Animated 3D humans built from Gaussian splats work by attaching small colored blobs to a standard body mesh: SMPL, a fixed template of 6,890 vertices. The attached blobs move with the mesh's skeleton for free. This makes such avatars cheap to transmit, because each frame needs only a few dozen pose numbers. It also makes them cheap to render. The catch is that the template has the topology of a naked body: two separate legs, a closed torso. A skirt spanning both legs, or an open jacket flapping away from the chest, cannot be represented. Methods that let blobs drift freely fix the shape but give up both savings. Every free blob must be sent each frame. Free blobs also pile up in a few screen regions, which stalls parallel rendering hardware.
This note proposes treating the places where clothing departs from the body template as a scarce, explicitly priced resource. The avatar keeps most of its blobs bound to the mesh. It is allowed a small population of "detached" blobs, with their own coarse motion graph, only on mesh regions where the template provably fails. How many detached blobs it gets, and where, is decided during training by an objective that charges for two things measured on the target device: the per-frame bits they add, and the worst-case per-screen-tile rendering load they create. The bet is that topological freedom, bitrate, and rendering stalls are all concentrated in the same few regions. If so, a representation that budgets them jointly should beat any pipeline that trains for quality first and compresses and schedules afterwards.
Core Claim
The proposal makes three linked claims.
The first is about where failures and costs sit. In loose-garment sequences, the regions where a mesh-bound avatar fails coincide with the regions that dominate two things in hybrid methods: per-frame bitrate and per-tile load imbalance.
The second is a structural point, and it is the new technical lever. Mesh-bound blobs come with a nearly free workload predictor. Rasterizing the coarse posed mesh into screen tiles before splatting tells the renderer how many blobs will land in each tile. Only detached blobs are unpredictable. So the representation can be shaped to keep the unpredictable part small and bounded.
The third is the falsifiable claim. Training with a hardware-derived tile-load penalty and a rate penalty yields a better quality–bitrate–tail-latency frontier than three alternatives: the same model trained for quality and then pruned and load-balanced post hoc; existing compact avatar codecs; and a plain penalty on the number of blobs.
How This Relates to Prior Work
Two lines of work show what fixed topology costs. Learning to Dress 3D People in Generative Clothing shows that vertex offsets on SMPL turn skirts into "pants-like" shrink-wrapped shapes. SMPLicit shows that open jackets need disjoint surface flaps a fixed mesh cannot form, and answers with an implicit distance field. These papers motivate the work. We do not adopt implicit fields, because the file notes that driving them in new poses requires expensive inversion.
The hybrid methods we build on all inherit the base mesh's topology:
- GoMAvatar binds Gaussians to SMPL faces and reports 30.37 dB PSNR on ZJU-MoCap at 3.63 MB per subject.
- LIFe-GoM decouples a low-resolution skinning mesh from dense appearance blobs.
- FlashAvatar adds a learned offset field for hair.
GoMAvatar and LIFe-GoM state that they fail on loose coats and skirts. FlashAvatar and StreamME show that large unconstrained offsets produce floating artifacts. Our design reuses mesh binding for the bulk of the body. It replaces global offsets with a localized, budgeted detached population.
On compression, three systems mark the frontier:
- Towards Efficient 3D Gaussian Human Avatar Compression sends only 94 SMPL parameters per frame and reaches under 0.20 Mbps on ZJU-MoCap. By construction, it cannot represent loose garments.
- DualGS drives about 180k skin Gaussians from about 15k "joint" Gaussians through nearest-neighbor anchoring. It compresses them with residual vector quantization, WebP lookup tables, and codebooks to about 350 KB per frame.
- HGC-Avatar reaches 29.82 dB at 0.40 MB per frame on THuman4.0, but needs about 1.5 minutes to decode each frame.
DualGS is the closest existing work. It already gives every blob a flexible motion graph. Our difference is that we spend that flexibility only where the body template fails. The rest of the avatar keeps the near-zero per-frame cost of the Yin et al. design.
On hardware, LS-Gaussian and SPLATONIC supply both the method and the constraint:
- LS-Gaussian shows that uneven tile workloads stall Gaussian rasterization. It fixes this with depth-based workload prediction and a load-distribution unit, reporting an average 5.41× speedup on Jetson AGX Orin, but only for static scenes.
- SPLATONIC shows that pixel-parallel rendering beats tile-based rendering only when primitives are sparse, and loses data sharing when they clump.
Both treat the scene as given. Neither changes the representation to make the workload predictable. Our proposal moves that lever into training. Static pruning and level-of-detail methods, CoverPruneGS and GoDe, provide the post-hoc compression steps used in our sequential baseline. CoverPruneGS's Euclidean voxel grouping is exactly the kind of rule that merges blobs across a skirt's two sides when the legs come together.
Research Program
The avatar has two populations.
Bound blobs live on the faces of an SMPL-X mesh and are skinned as in GoMAvatar. Their per-frame cost is the pose vector alone.
Detached blobs live on a small set of motion nodes. Each node is attached by a learned gate to a mesh face, but its position is free and regularized to deform as rigidly as possible, as DualGS does. Each frame, the nodes need an entropy-coded motion residual relative to their skinned prediction.
Training starts with a bound-only warmup. The gate is then initialized from the per-face photometric residual that persists across poses, since a skirt gap or open flap shows up as error the mesh cannot remove. After that, gates, node counts, and blob parameters are optimized jointly under the following objective:
$$ \mathcal{L} = \mathcal{L}_{\text{photo}} + \lambda_R \, \hat{R} + \lambda_T \, \tau^{-1}\log \sum_{k} e^{\tau n_k} $$
The terms are defined as follows:
- $\mathcal{L}_{\text{photo}}$ is the usual image loss (L1 plus structural similarity).
- $\hat{R}$ is the estimated bits per frame for detached-node motion, from a learned entropy model of the residuals.
- $n_k$ is the number of blob–tile intersections in screen tile $k$, computed with a differentiable soft footprint.
- $\tau$ sets how closely the log-sum-exp approximates the maximum tile load.
The last term penalizes the most overloaded tile, not the total blob count. The most overloaded tile is what stalls a tile-parallel rasterizer, according to LS-Gaussian's analysis.
At render time, a lightweight scheduler runs before splatting. It rasterizes the coarse posed mesh into tiles to predict the bound-blob load, adds a per-node bound on detached load, and redistributes tiles across GPU thread blocks. This is the commodity-GPU analogue of LS-Gaussian's load-distribution unit. Implementing it as a CUDA kernel on Jetson AGX Orin is the main hardware deliverable. As a secondary check, a cycle-level model of an LS-Gaussian-style accelerator extended with a skinning stage will test whether the predictability argument carries over to dedicated silicon. This is not a tape-out.
Pruning and grouping of detached blobs use geodesic distance on the mesh plus node identity, not Euclidean voxels. Two sides of a skirt therefore never merge just because the knees touch.
Baselines
The naive reference point is a bound-only avatar with no detached blobs, at the per-frame cost of the Yin et al. framework. It shows how much the garment regions actually lose. On ZJU-MoCap it should nearly match the method, since that data is mostly tight clothing. The reference figure is the under-0.20 Mbps bitrate reported by Yin et al..
The strongest existing compression competitor is DualGS, which handles loose motion with flexible anchoring and reports about 350 KB per frame on its own capture rig, a dataset we do not have. We will retrain it on our sequences with its published codec stages. It gets the same number of training iterations and the same hyperparameter search budget as our method, and its rate is swept by varying codebook size. Beating it means lower garment-region perceptual error (LPIPS) at equal bits per frame, together with lower 99th-percentile frame time on Orin.
HGC-Avatar is reported, not chased, on THuman4.0: 29.82 dB at 0.40 MB per frame, with 1.5-minute decoding. Matching its quality at real-time decode would count as a clear win.
The sequential co-design baseline is the decisive competitor. It is our own two-population model trained with only the photometric loss. CoverPruneGS-style coverage pruning and GoDe-style quantization-aware truncation are then applied to the same bitrate. Finally, it is rendered with our runtime scheduler. It shares data, architecture, iteration count, and scheduler with our method, and differs only in whether costs enter training.
Two ablations isolate the key ingredients. One replaces the tile-load term with a plain penalty on total detached-blob count. The other removes the residual-based gate initialization and lets detached blobs appear anywhere.
ActorsHQ and AvatarRex will be our main loose-garment test sets. No reported result was found on Lacuna for any baseline's garment-region quality on ActorsHQ or AvatarRex, so all those numbers will come from our reruns.
Why not just…?
Why not just learn vertex offsets on SMPL, or a FlashAvatar-style global offset field? Offsets on a fixed mesh cannot span the gap between legs, as CAPE documents. Global offset fields drift into floaters under novel poses, as FlashAvatar and StreamME report. We include the offset-field variant as a baseline, trained under the same rate budget, to measure this on our garment sequences.
Why not just use DualGS, which already frees every blob? This is a real threat. If DualGS's nearest-neighbor anchoring already reaches our quality at our bitrate, our design adds nothing. Our argument is that DualGS pays motion bits and unpredictable tile load for the whole body, including the roughly 80–90% that the body template explains well. The matched-rate, matched-latency comparison on loose-garment sequences settles this directly.
Why not just train for quality and then prune, quantize, and load-balance at runtime? This is the standard decoupled workflow, and it is the hypothesis most likely to sink the proposal. If the sequential baseline lands within noise of our method on the quality–bitrate–tail-latency frontier, the co-design claim is false. We would then report the two-population representation alone as the contribution.
Why not just penalize blob count? Count is a proxy for total work, but stalls come from the busiest tile, and clumped detached blobs can keep the count low while overloading one tile. The count-penalty ablation tests whether this distinction matters in practice.
Proposed Experiments
We will train subject-specific avatars on ZJU-MoCap as the tight-clothing sanity set, THuman4.0 for comparison with HGC-Avatar, and a hand-selected loose-garment split of ActorsHQ and AvatarRex (skirts, coats, open jackets). On the loose-garment split, we will annotate garment masks so error can be measured where topology breaks, separately from whole-image averages.
Metrics are PSNR, SSIM, and LPIPS, both full-frame and inside garment masks; bits per frame; median and 99th-percentile frame time on Jetson AGX Orin and on a desktop GPU; and the ratio of maximum to mean tile load.
The first decisive check is the sanity case. On ZJU-MoCap the learned gates should close almost entirely and the bitrate should approach the Yin et al. figure. If many detached blobs survive on tight clothing, the gating is fitting noise.
The second check is the matched-cost comparison on the loose-garment split. Our method should reduce garment-region LPIPS relative to both DualGS and the sequential baseline at equal bits and equal 99th-percentile latency.
The third check is the mechanism. The ratio of maximum to mean tile load should fall under the tile-load penalty but not under the count penalty. The share of motion bits spent on detached nodes should track the annotated garment area, which would confirm that cost and topology failure are concentrated in the same places.
References
- Learning to Dress 3D People in Generative Clothing
- SMPLicit: Topology-Aware Generative Model for Clothed People
- GoMAvatar: Efficient Animatable Human Modeling from Monocular Video Using Gaussians-on-Mesh
- LIFe-GoM: Generalizable Human Rendering with Learned Iterative Feedback Over Multi-Resolution Gaussians-on-Mesh
- FlashAvatar: High-Fidelity Head Avatar with Efficient Gaussian Embedding
- StreamME: Simplify 3D Gaussian Avatar within Live Stream
- Towards Efficient 3D Gaussian Human Avatar Compression: A Prior-Guided Framework
- Robust Dual Gaussian Splatting for Immersive Human-centric Volumetric Videos
- HGC-Avatar: Hierarchical Gaussian Compression for Streamable Dynamic 3D Avatars
- No Redundancy, No Stall: Lightweight Streaming 3D Gaussian Splatting for Real-time Rendering
- SPLATONIC: Architectural Support for 3D Gaussian Splatting SLAM via Sparse Processing
- CoverPruneGS: Coverage-Preserving Structured Pruning for Compact 3D Gaussian Splatting from Sparse-View Monocular Videos
- GoDe: Gaussians on Demand for Progressive Level of Detail and Scalable Compression