ARG-Avatar: Animation-Rendering Guidance for Efficient Feedforward Avatar Reconstruction

Jianjin Xu, Soham Dasgupta, Fernando De la TorrearXiv 2026 · arXiv

TL;DR

ARG-Avatar is a highly efficient feedforward framework for reconstructing animatable 3D human avatars from a few images. By explicitly integrating geometric priors—like animation and rendering—directly into the model's attention mechanism, it achieves state-of-the-art quality with only 68M parameters, outperforming models nearly ten times its size.

What Problem Does This Paper Address?

Creating high-quality, controllable 3D avatars typically requires either lengthy per-person optimization or massive "scaling-first" feedforward models. Recent state-of-the-art methods (like the LHM series) rely on hundreds of millions of parameters and hundreds of thousands of private video sequences to learn how to map 2D image pixels to 3D avatar parts.

The authors argue that this brute-force scaling is unnecessary. Existing methods force neural networks to "re-learn" basic geometric correspondences that are already mathematically well-defined by human body templates and camera projection. ARG-Avatar aims to bridge this efficiency gap by using Animation-Rendering Guidance to help the model focus on the right parts of the input images from the start.

Figure 1. Overview of ARG-Avatar. Given casual photographs or frames from a monocular video, ARG-Avatar reconstructs a canonical avatar and produces avatars with pose-dependent deformations for arbitrary driving poses. ARG-Avatar has only 68M trainable parameters.

Methods

ARG-Avatar represents avatars using 3D Gaussian Splatting (3DGS) anchored to a standard human mesh template. The core of the architecture consists of two novel components:

  • FACRoPE (Foreground Avatar Coordinate RoPE): Standard attention mechanisms are "geometry-blind," searching the entire image for features. FACRoPE uses the known body pose and camera geometry to project each 3D avatar token into the 2D image plane. These coordinates are then used in a Rotary Position Embedding (RoPE) framework, effectively telling each token exactly where it should "look" in the input image.
  • ITR (Intermediate Token Rendering): This is a "rendering-in-the-loop" strategy. The network decodes its intermediate features into a coarse 3D avatar, renders it, and uses that projected geometry to guide subsequent layers. This iterative refinement ensures that the model's attention remains grounded in 3D space.
  • Pose-Dependent Deformation: To handle non-rigid effects like loose clothing or hair, the model predicts residuals that adjust the canonical avatar based on the target driving pose.
FACRoPE (top) produces sharp, precise attention maps compared to the diffuse, often incorrect attention of naive mechanisms (bottom).

Datasets

Unlike many competitors that rely on massive proprietary datasets, ARG-Avatar is trained exclusively on publicly available datasets:

  • Training: A combination of DNARendering, 2K2K, 4D-Dress, Human4DiT, THuman2.1, and HumanVid, totaling approximately 6.8K subjects and 64K sequences.
  • Evaluation: The primary benchmark is the NeuMan dataset, used to test out-of-distribution performance on "in-the-wild" captures.

Key Insights and Ideas

  • Geometry is a Strong Prior: You don't need a billion parameters to learn that a hand in 3D corresponds to a hand in a 2D photo if you already have the math for Linear Blend Skinning (LBS) and camera projection.
  • Disentanglement through Regularization: The authors introduced a "Common Shell Transfer" regularizer. This forces the model to capture shared geometric traits in the canonical (rest) pose rather than redundantly encoding them in every individual pose-dependent deformation.
  • Efficiency as a Feature: By reducing the search space for attention, the model can be much smaller and faster without sacrificing (and actually improving) reconstruction quality.

Key Results

ARG-Avatar sets a new bar for efficiency and quality in feedforward reconstruction:

  • Superior Quality: It achieves a 30.82 dB PSNR on the NeuMan benchmark, a 2.7 dB improvement over the previous state-of-the-art (LHM++).
  • Massive Efficiency Gains: Compared to LHM++, it uses 9× fewer trainable parameters (68M vs. 609M), requires 14× less training compute, and runs 6× faster at inference time (2.3s vs. 14.2s for a 6-view reconstruction).
  • Data Efficiency: It achieves these results using only ~21% of the training data sequences used by its closest competitors.
ARG-Avatar (red) achieves higher PSNR on the NeuMan dataset with significantly less training compute and fewer parameters than baselines.

Limitations and Future Work

  • Single-View Performance: While ARG-Avatar excels with two or more images, its performance drops with only a single input view. The authors note it is not currently trained to "inpaint" or hallucinate invisible regions (like the back of a person).
  • Template Dependency: The system relies on a predefined human template (MHR/SMPL-X). While this provides the necessary geometric guidance, it may limit the model's ability to reconstruct extreme body shapes or very bulky clothing that deviates significantly from the template.
  • Future Directions: Integrating generative priors (like diffusion models) could help the system handle single-view inputs more gracefully by hallucinating missing details.

Summary updated

Abstract

Reconstructing animatable 3D human avatars from images is important for applications such as gaming and augmented reality. Feedforward methods make avatar reconstruction accessible by reducing inference time to a few seconds, but recent approaches rely heavily on scaling model capacity, training data, and computation. We ask whether such scaling is necessary. To this end, we introduce ARG-Avatar, an efficient feedforward framework with only 68M trainable parameters that achieves SOTA reconstruction quality. Our key insight is that 3D animation and rendering already provide strong geometric correspondences that existing methods largely attempt to learn from data. In light of this, we design the Foreground Avatar Coordinate RoPE attention (FACRoPE) and Intermediate Token Rendering (ITR) pipeline to guide avatar tokens to attend to their corresponding patches. Experiments across multiple datasets show that ARG-Avatar consistently outperforms prior methods when using two or more input images. Compared with LHM++, ARG-Avatar uses approximately 9$\times$ fewer trainable parameters, 4.8$\times$ fewer training sequences, and 14$\times$ less training compute, while running 6$\times$ faster and achieving a 2.7 dB PSNR improvement on the out-of-distribution NeuMan benchmark.
BibTeX
@misc{xu2026arg,
  author = {Jianjin Xu and Soham Dasgupta and Fernando De la Torre},
  title = {{ARG-Avatar: Animation-Rendering Guidance for Efficient Feedforward Avatar Reconstruction}},
  year = {2026},
  eprint = {2611.22207},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.22207},
}
Figure