Introduction
Neural Radiance Fields (NeRFs) have revolutionized the creation of photorealistic 3D digital humans. However, current state-of-the-art methods typically rely on per-identity optimization, requiring a dedicated model for every individual. This "one model per person" paradigm creates a significant scaling bottleneck: modeling 100 people requires 100 times the training resources. Furthermore, these specialist models often fail to generalize to expressions not present in a specific subject's training video. MI-NeRF (Multi-Identity NeRF) addresses these challenges by learning a single, unified 4D facial model capable of representing hundreds of identities simultaneously.
The Multiplicative Module
The core innovation of MI-NeRF is its architectural approach to disentangling identity from expression. Traditional multi-subject models often use simple concatenation of identity and expression vectors, which can lead to "identity leaking"—where the synthesized face appears as a blend of multiple people.
MI-NeRF introduces a Multiplicative Module ($M$) inspired by multilinear factor analysis. Instead of concatenation, the module computes a non-linear mapping based on the element-wise (Hadamard) product of identity and expression features. This provides a strong inductive bias, allowing the network to learn how specific expressions deform different facial structures. This module feeds into a unified dynamic NeRF backbone that predicts volume density and color based on 3D coordinates and viewing directions.
Experimental Setup and Datasets
To train a robust "generalist" model, the authors curated a diverse dataset of 140 high-definition talking face videos. This collection includes standard benchmarks like the Obama dataset, the PATS dataset (featuring presenters like Trevor Noah), and the LipNeRF dataset (featuring actors like Al Pacino).
The model is conditioned on 3D Morphable Model (3DMM) parameters for expression, learned time-invariant identity codes, and per-frame latent codes to capture fine-grained details such as lighting changes. Of the 140 videos, 100 were used for training the base model, while the remainder served to test generalization to unseen identities.
Results and Efficiency
MI-NeRF demonstrates remarkable efficiency gains. Training a single model for 100 identities takes approximately 80 hours, a 90% reduction compared to the ~4,000 hours required to train 100 individual NeRF models.
Beyond speed, the model excels in expression transfer. Because it learns from a collective "dictionary" of facial movements, it can synthesize expressions for a target subject that were never seen in that subject's original video. For instance, a smile learned from one person can be accurately mapped onto another.
Quantitative evaluations show that MI-NeRF maintains identity better than single-identity baselines (like NeRFace) while producing fewer artifacts during extreme expression synthesis. Additionally, the model can be "personalized" to a completely new individual using as little as 1-3 seconds of video, significantly lowering the barrier for high-quality digital human creation.
Conclusion and Future Work
MI-NeRF represents a significant step toward scalable, high-fidelity digital humans. By moving from per-identity specialists to a unified generalist model, it achieves sublinear scaling and improved robustness. While currently dependent on 3DMM accuracy and limited in its handling of external factors like hair movement, the framework provides a foundation for future models trained on thousands of identities from massive "in-the-wild" video collections.
Summary updated