One Code per Material: Neural Representation and Generation of Measured BTFs

Fazilet Gokbudak, Canberk Baykal, Chenliang Zhou, Rafal Mantiuk, Cengiz OztireliarXiv 2026 · arXiv

This paper from the University of Cambridge presents a significant leap in how we represent and generate complex real-world materials. By moving from massive, per-pixel data structures to a "one code per material" approach, the authors enable high-fidelity material reconstruction and the generation of entirely new, physically grounded textures.

TL;DR

The authors introduce a neural representation for Bidirectional Texture Functions (BTFs) that compresses a complex material's appearance into a single 256-float code. This compact approach not only sets new state-of-the-art benchmarks for reconstruction quality but also enables a generative flow model to sample entirely new, relightable materials from a learned latent space.

What Problem Does This Paper Address?

Capturing the "soul" of a material—the way light dances across carpet loops, shadows fall into leather cracks, or highlights shift with parallax—requires Bidirectional Texture Functions (BTFs). Unlike standard analytic models (SVBRDFs) used in most game engines, BTFs capture complex effects like self-shadowing and inter-reflections by measuring a sample under thousands of light and view combinations.

The challenge is twofold:

  1. Size: A single measured material can consist of billions of values, reaching gigabytes in size.
  2. Generativity: Existing neural BTF representations use "per-texel latent maps" (essentially images of features). While these compress the data, they don't allow you to easily "sample" a new material. You can't just pick a random feature map and expect it to look like a coherent, realistic material.

Methods

The researchers propose a two-tier neural architecture to solve these issues:

  • Per-Material Coordinate Network: For maximum accuracy, they fit a Fourier-feature MLP to a single material. This network maps a texel position $(x)$ and a light/view direction pair (represented by Rusinkiewicz vectors) directly to an RGB value.
  • The Compact Model: To enable a shared space, they replace the per-material network with a single, massive Shared Decoder ($g_\theta$) conditioned on a global 256-float code ($z_m$). This reduces the material-specific data from millions of features to just 256 numbers.
  • Set Encoder: To find the right code for a material, they use a permutation-invariant encoder that processes a small set of random samples from a measurement.
  • Rectified Flow for Generation: Because every material is now just a point in a 256-dimensional space, the authors can train a class-conditional rectified flow model to sample new points. These sampled codes can then be "rendered" by the shared decoder into full, relightable BTFs.
  • Colourway Transfer: To combat data scarcity, they developed a method to transfer the color characteristics of one material onto the physical structure of another within the same class, effectively augmenting their training set.
Figure 2. Overview. The compact model encodes each measured material and its colourways (Sec. 3.3) into one 256-float code with a set encoder; a decoder gθ shared by all materials maps the code, texel position and light and view directions to RGB. A class-conditional rectified flow fitted to these codes samples codes, which the same decoder renders as full BTFs that can be relit. Top right: measured materials reconstructed by the compact model, rendered on objects.

Datasets

  • UBO2014: The gold standard for measured BTFs, containing 84 materials across seven classes (carpet, fabric, felt, leather, stone, wallpaper, and wood).
  • Augmented Dataset: Using their "colourway" technique and hue rotations, the authors expanded the 84 base captures into a dataset of 886 materials to provide enough variety for the generative flow model to learn.
Figure 3. Colourways. One measured material per row (left) and four of its colourways: its structure in the colours of other mate- rials of its class, as flash photographs. Every value is a measured value under one 3×3 colour transform.

Key Insights and Ideas

  • Global vs. Local Latents: The "aha moment" is realizing that a material's identity can be captured globally. Moving from per-texel maps to a single code treats a material as a single entity in a vector space, which is the prerequisite for true generative modeling.
  • Measured Realism in Generative Space: Most material generators learn from flat photos or synthetic data. This model learns from measured light transport. When the model "hallucinates" a new fabric, it isn't just dreaming up a pattern; it's generating a structure that inherently understands how to cast shadows on itself.
  • The Kilobyte Scale: The compact model brings measured BTFs down to the kilobyte scale per material, making them practical for complex scenes with hundreds of unique surfaces.

Key Results

  • Superior Reconstruction: The per-material model outperforms the previous state-of-the-art (Xia et al.) by +3.2 dB PSNR. Even the highly compressed compact model beats prior per-texel methods (like Rainer et al.) by +1.2 dB PSNR while using 14x fewer values per material.
  • Rate-Distortion Efficiency: As shown in Figure 4, the compact model achieves a much better balance between storage size and visual quality than any previous baseline.
  • Generative Quality: The flow model achieved the lowest Kernel Inception Distance (KID) and Maximum Mean Discrepancy (MMD) compared to image-based generators, proving that it produces more realistic and diverse samples.
Figure 4. Rate–distortion on nine UBO2014 materials: PSNR against the number of values stored per material (log scale). The compact model is shown with its 256-float code and with the shared decoder amortised over 84 materials.

Limitations and Future Work

  • Structural Novelty: Because the model is trained on only 84 unique physical structures, its generative "creativity" is mostly limited to new color combinations and subtle blends. It doesn't yet "invent" entirely new weave patterns.
  • Bake-out Time: To use these materials in a real-time renderer, the neural model must be decoded into a light-view table, which currently takes about 3.7 minutes on an NVIDIA A100.
  • Tiling: The model represents a fixed-size measured patch. Future work could integrate tiling and synthesis techniques to cover larger surfaces.

The authors hope this work makes measured appearance a practical target for generative modeling, bringing us one step closer to digital environments that are indistinguishable from reality.

Summary updated

Abstract

Measured bidirectional texture functions (BTFs) capture the full appearance of real materials--including parallax, self-shadowing and inter-reflections that analytic SVBRDFs cannot express--but each measurement is billions of values, and every new material requires a new capture. Existing neural BTF representations encode each material as large per-texel latent maps rather than as a point in a shared space from which new materials could be sampled. We present neural representations of measured BTFs at two operating points. A per-material coordinate network outperforms the recent neural BTF of Xia et al. at matched size (+3.2 dB PSNR) and all prior baselines on every metric. A compact model represents each material by a single 256-dimensional code that conditions a shared decoder; it exceeds the PSNR (+1.2 dB) and colour accuracy of the per-texel latent maps of Rainer et al. with 256 numbers per material instead of millions, and one decoder shared by all materials. Because each material is a single point in this code space, measured BTFs can be sampled: a class-conditional flow over the codes, trained on the 84 materials enlarged with colourways and hue rotations, generates materials that render and relight as full BTFs.
BibTeX
@misc{gokbudak2026one,
  author = {Fazilet Gokbudak and Canberk Baykal and Chenliang Zhou and Rafal Mantiuk and Cengiz Oztireli},
  title = {{One Code per Material: Neural Representation and Generation of Measured BTFs}},
  year = {2026},
  eprint = {2611.20948},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20948},
}
Figure