Domain-Tuned Diffusion Super-Resolution for 4K Facial Asset Maps

Elly Akhoundi, Mathieu Lamarre, Thibault Goyallon, Patrick Anderson, Étienne DanvoyearXiv 2026 · arXiv

TL;DR

This paper introduces a domain-tuned diffusion upscaler that transforms 1K facial asset maps into production-ready 4K textures. By adapting a frozen Stable Diffusion XL (SDXL) backbone with lightweight LoRA modules and a Tile ControlNet, the authors achieve high-fidelity results for both albedo and displacement maps without needing a custom super-resolution architecture.

What Problem Does This Paper Address?

In high-end digital human workflows (like cinematic VFX or AAA games), facial assets require 4K resolution to look realistic in close-ups. These assets typically consist of albedo maps (surface color) and displacement maps (fine geometric details like pores and wrinkles). However, many generative pipelines only produce these maps at 1K resolution.

Simply using off-the-shelf photographic super-resolution (SR) models doesn't work well for two reasons:

  1. Domain Mismatch: Facial UV maps have unique statistics—they contain seams, charts, and non-photographic layouts that generic models aren't trained for.
  2. Geometric Integrity: Displacement maps are scalar height fields. Generic SR models often "hallucinate" visually pleasing but geometrically incorrect details, which can lead to distorted or unrealistic facial geometry when rendered.
Comparison of 1K vs. 4K albedo and displacement maps.

Methods

Instead of building a new model from scratch, the researchers used a "domain recipe" to adapt a powerful pretrained model:

  • Backbone: They started with a frozen SDXL latent-diffusion model.
  • Adaptation: They added lightweight LoRA (Low-Rank Adaptation) modules to the UNet and a Tile ControlNet. The LoRA modules specialize the model for the specific "look" of facial assets, while the Tile ControlNet ensures the upscaled version maintains the exact structure of the 1K input.
  • Modality-Specific Handling: They trained separate models for albedo and displacement.
  • For displacement, they developed a specific formulation to handle scalar data within an RGB-focused diffusion pipeline, ensuring the geometric range is preserved.
  • For albedo, they implemented a post-processing step in the CIELAB color space to "anchor" the final colors to the original low-resolution input, preventing the diffusion model from drifting away from the intended skin tone.
The training architecture using SDXL, LoRA, and Tile ControlNet.

Datasets

The authors evaluated their method on 100 held-out maps that were not seen during training. These maps were captured using a high-end 28-camera 12MP rig, providing a ground-truth (GT) 4K reference to compare against. The training data likely came from a similar internal high-resolution capture pipeline.

Key Insights and Ideas

  • Recipe over Architecture: You don't need a custom-built super-resolution network to get state-of-the-art results. Targeted domain adaptation of a strong general-purpose model (like SDXL) is more efficient and effective.
  • Structure vs. Detail: The 1K input doesn't contain enough information to uniquely determine every pore. Therefore, the goal isn't "pixel-exact" reconstruction, but rather structural consistency. The model restores plausible, domain-correct micro-detail that "fits" the low-res structure.
  • Generic Priors are Harmful: Off-the-shelf models like Real-ESRGAN or standard Stable Diffusion upscalers actually performed worse than simple bilinear upsampling in some metrics because their "natural image" priors were so mismatched for the UV-map domain.

Key Results

The proposed method significantly outperformed both simple interpolation and popular AI upscalers:

  • Albedo: Reduced the perceptual error (LPIPS) from 0.21 (bilinear) to 0.08. Qualitative results show it avoids the "grainy" or "flat" look of other methods.
  • Displacement: Recovered 97% of the ground-truth detail energy, compared to only 46% for bilinear upsampling. Unlike generic models, it didn't overshoot or misplace the relief of wrinkles and pores.
  • Visual Fidelity: Renders using the 4K maps produced by this method were much closer to the ground-truth captures than any baseline.

Qualitative comparison of albedo super-resolution results. Qualitative comparison of displacement super-resolution results.

Limitations and Future Work

  • Single Source: The evaluation was limited to a single capture pipeline. It's unclear how well the model generalizes to different scan qualities or different upscaling factors (e.g., 8x).
  • Consistency: The models for albedo and displacement are separate. While they share a UV layout, there is no explicit mechanism to ensure that a pore in the albedo map perfectly aligns with a pore in the displacement map.
  • Scope: The current work doesn't address other important maps like specular, roughness, or normal maps.
  • Human in the Loop: The authors note that for high-end cinematic VFX, an artist's review is still necessary, as the AI-generated micro-structure is "hallucinated" (albeit plausibly).

Summary updated

Abstract

Physically based facial assets require 4K albedo and displacement maps for cinematic close-ups, yet generative pipelines commonly produce them at only 1K. Naively applying photographic super-resolution is a poor fit: facial UV maps have domain-specific statistics, and displacement is a scalar height field whose errors directly distort geometry. We present a domain-tuned diffusion upscaler for 4x (1024 to 4096) facial asset super-resolution. Starting from a bilinearly upsampled map, we adapt a frozen SDXL backbone per modality with lightweight LoRA modules, while a Tile ControlNet preserves input structure as the diffusion prior restores pore- and wrinkle-scale detail. On held-out maps, our method produces sharper, more faithful detail than bilinear and off-the-shelf GAN and diffusion super-resolvers, reducing albedo LPIPS from 0.21 to 0.08 while avoiding misplaced or excessive displacement relief. These results show that lightweight domain adaptation can produce render-ready 4K facial asset maps without a custom super-resolution architecture.
BibTeX
@misc{akhoundi2026domain,
  author = {Elly Akhoundi and Mathieu Lamarre and Thibault Goyallon and Patrick Anderson and Étienne Danvoye},
  title = {{Domain-Tuned Diffusion Super-Resolution for 4K Facial Asset Maps}},
  year = {2026},
  eprint = {2611.21890},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21890},
}
Figure