TL;DR
This paper introduces a domain-tuned diffusion upscaler that transforms 1K facial asset maps into production-ready 4K textures. By adapting a frozen Stable Diffusion XL (SDXL) backbone with lightweight LoRA modules and a Tile ControlNet, the authors achieve high-fidelity results for both albedo and displacement maps without needing a custom super-resolution architecture.
What Problem Does This Paper Address?
In high-end digital human workflows (like cinematic VFX or AAA games), facial assets require 4K resolution to look realistic in close-ups. These assets typically consist of albedo maps (surface color) and displacement maps (fine geometric details like pores and wrinkles). However, many generative pipelines only produce these maps at 1K resolution.
Simply using off-the-shelf photographic super-resolution (SR) models doesn't work well for two reasons:
- Domain Mismatch: Facial UV maps have unique statistics—they contain seams, charts, and non-photographic layouts that generic models aren't trained for.
- Geometric Integrity: Displacement maps are scalar height fields. Generic SR models often "hallucinate" visually pleasing but geometrically incorrect details, which can lead to distorted or unrealistic facial geometry when rendered.
Methods
Instead of building a new model from scratch, the researchers used a "domain recipe" to adapt a powerful pretrained model:
- Backbone: They started with a frozen SDXL latent-diffusion model.
- Adaptation: They added lightweight LoRA (Low-Rank Adaptation) modules to the UNet and a Tile ControlNet. The LoRA modules specialize the model for the specific "look" of facial assets, while the Tile ControlNet ensures the upscaled version maintains the exact structure of the 1K input.
- Modality-Specific Handling: They trained separate models for albedo and displacement.
- For displacement, they developed a specific formulation to handle scalar data within an RGB-focused diffusion pipeline, ensuring the geometric range is preserved.
- For albedo, they implemented a post-processing step in the CIELAB color space to "anchor" the final colors to the original low-resolution input, preventing the diffusion model from drifting away from the intended skin tone.
Datasets
The authors evaluated their method on 100 held-out maps that were not seen during training. These maps were captured using a high-end 28-camera 12MP rig, providing a ground-truth (GT) 4K reference to compare against. The training data likely came from a similar internal high-resolution capture pipeline.
Key Insights and Ideas
- Recipe over Architecture: You don't need a custom-built super-resolution network to get state-of-the-art results. Targeted domain adaptation of a strong general-purpose model (like SDXL) is more efficient and effective.
- Structure vs. Detail: The 1K input doesn't contain enough information to uniquely determine every pore. Therefore, the goal isn't "pixel-exact" reconstruction, but rather structural consistency. The model restores plausible, domain-correct micro-detail that "fits" the low-res structure.
- Generic Priors are Harmful: Off-the-shelf models like Real-ESRGAN or standard Stable Diffusion upscalers actually performed worse than simple bilinear upsampling in some metrics because their "natural image" priors were so mismatched for the UV-map domain.
Key Results
The proposed method significantly outperformed both simple interpolation and popular AI upscalers:
- Albedo: Reduced the perceptual error (LPIPS) from 0.21 (bilinear) to 0.08. Qualitative results show it avoids the "grainy" or "flat" look of other methods.
- Displacement: Recovered 97% of the ground-truth detail energy, compared to only 46% for bilinear upsampling. Unlike generic models, it didn't overshoot or misplace the relief of wrinkles and pores.
- Visual Fidelity: Renders using the 4K maps produced by this method were much closer to the ground-truth captures than any baseline.

Limitations and Future Work
- Single Source: The evaluation was limited to a single capture pipeline. It's unclear how well the model generalizes to different scan qualities or different upscaling factors (e.g., 8x).
- Consistency: The models for albedo and displacement are separate. While they share a UV layout, there is no explicit mechanism to ensure that a pore in the albedo map perfectly aligns with a pore in the displacement map.
- Scope: The current work doesn't address other important maps like specular, roughness, or normal maps.
- Human in the Loop: The authors note that for high-end cinematic VFX, an artist's review is still necessary, as the AI-generated micro-structure is "hallucinated" (albeit plausibly).
Summary updated