Animatable human avatars built from 3D Gaussians, which represent a person as hundreds of thousands of small colored blobs, recompute every blob's position offset, color offset, rotation, scale and opacity each frame from the current body pose. With one avatar on a desktop GPU this is cheap. With dozens of avatars on one GPU, as in a telepresence server or a crowded game scene, these per-frame attribute buffers become the main churning memory in the system. Avatars join and leave and switch level of detail, so buffers of different sizes are allocated and freed continuously. The rasterizer then reads many attribute arrays per blob, scattered across memory. This proposal borrows a database idea to shrink and reorganize those buffers. Many attributes are largely predictable from a few others. The pose-dependent position offset tracks the skinned surface normal, and the color offset tracks shading, which GoMAvatar already exploits by hand. Predictable attributes need not be stored at all. They can be recomputed inside the rendering kernel from a small set of stored reference attributes plus sparse corrections.
The database version of this idea has a known cost. Reading a derived column means also fetching its reference columns, and Corra measures up to a 1.66x latency penalty for this. Our central bet is that this penalty is a memory-placement problem. If the choice of which attribute predicts which is made together with where attributes sit in memory, so that each derived attribute's references share its page-aligned block, then the compression win survives at render time. The result is a per-avatar buffer of nearly constant size, which can be placed by a simple allocation plan instead of a general-purpose allocator.
Core Claim
Compressing per-frame pose-dependent Gaussian attributes can reduce both the bytes the rasterizer reads and allocator fragmentation under avatar churn, without the fetch penalty that correlation-aware compression pays in databases. This requires choosing the dependency structure among attributes with a layout-aware cost, not a size-only cost. In this scheme, "derived" (virtual) attributes are computed inside the rasterizer from reference attributes that sit in the same memory group, and only reference attributes and sparse residuals are ever materialized.
What is new is the coupling. Correlation-aware compressors choose dependencies purely to minimize storage. Avatar systems treat memory purely as total capacity. Allocators treat the data as opaque bytes. We make the dependency graph decide the memory layout, and the layout constrain the dependency graph.
Where This Sits in Prior Work
The method comes from tabular compression. Lightweight Correlation-Aware Table Compression (Virtual) fits sparse cluster-wise linear models between columns. It stores only residuals and outliers, and picks an acyclic set of dependencies greedily, reaching up to 40% smaller files than Parquet with Snappy while scanning fast. Corra adds diff and rule encodings and reports the fetch penalty above. Stoian and Kobiolka drop exactly derivable columns and rebuild them at query time. We reuse Virtual's regression-with-residuals machinery almost unchanged. Our departure is the objective used to pick dependencies, and the fact that decoding happens inside a GPU kernel rather than a scan operator.
Two graphics works motivate the proposal by showing cross-attribute redundancy is real in Gaussian representations. ICGS-Quantizer quantizes rotation, scale and opacity jointly in voxel blocks and beats per-attribute compressors (16.9 KB vs. 21.1 KB for C3DGS on Google Scanned Objects). It targets static scenes, not articulated poses, and it decodes offline. MEGA replaces 144 spherical-harmonic color parameters in dynamic scenes with a 3-parameter base color plus a shared small predictor. It conditions on time rather than on a skeleton and measures only storage.
The avatar models supply the application. GaussianAvatar predicts per-frame position and color offsets from a pose encoder. ParDy-Human refines skinned positions with a per-frame deformation module. GoMAvatar anchors Gaussians to mesh faces and derives shading from normals, reaching 3.63 MB per subject. GoMAvatar is the closest existing work. It hand-designs one cross-attribute dependency, shading from normals. We instead mine which attributes are derivable per avatar and per body region, and we tie that choice to memory.
The memory side both motivates and constrains the proposal. STAlloc shows that when allocation sizes and lifetimes are regular, an offline plan cuts GPU fragmentation by 85.1% on average. It also notes that irregular workloads fall back to weaker heuristics, and avatar churn is irregular unless buffer sizes are made predictable. vAttention reserves contiguous virtual ranges and maps physical pages on demand, which lets kernels keep simple contiguous indexing. We reuse this for per-avatar regions. TEMERAIRE shows that generic allocators under churn lose huge-page coverage, since only 11.8% of heap was hugepage-backed before their packing allocator. That cautions against assuming default mechanisms suffice. The TLB evidence, up to 10.9% of runtime in Dissecting Scale-Out Applications Performance on Diverse TLB Designs and up to 50% in Towards Faster Reasoners by Using Transparent Huge Pages, comes entirely from CPUs. Whether address translation matters for GPU splatting is therefore an open question we must measure, not assume.
Research Program
We first turn each trained avatar into a table. Taking GoMAvatar and GaussianAvatar models trained on a subject, we run them over the training and held-out poses and record, for every Gaussian and pose, the full attribute vector. Candidate reference columns include cheap quantities: the skinned canonical position, the skinned mesh normal, dominant bone weights, and a low-dimensional pose code. Candidate dependent columns are the pose-dependent offsets, rotation, scale, color and opacity. Running Virtual's regression search on this table tells us, per body region, how much of each dependent attribute is linearly predictable and how large the residuals are. This profiling step is itself a result. If loose clothing regions on THUman4.0 turn out to be unpredictable, they simply stay materialized.
Next we define layout-aware dependency selection. Attributes are packed into column groups, each assigned to one page-aligned block of a per-avatar region. The page granularity is 64 KB or 2 MB, chosen through CUDA virtual memory mapping as in vAttention. For a candidate dependency graph $G$ and group assignment $\pi$, we minimize
$$ C(G,\pi) = \sum_{j} b_j(G) + \lambda \sum_{(i \to j) \in G} \left[\pi(i) \neq \pi(j)\right] n_{ij}, $$
where $b_j(G)$ is the bits materialized for attribute $j$ (full column if stored, residuals and outliers if derived), $[\cdot]$ is 1 when reference $i$ and dependent $j$ land in different groups, $n_{ij}$ is how many rasterizer reads that split adds per frame, and $\lambda$ trades bytes against scattered reads. Setting $\lambda = 0$ recovers Virtual's size-only objective. We keep a greedy acyclic search like Virtual's, adding group reassignment moves, and sweep $\lambda$.
Third, we modify a Gaussian rasterizer so derived attributes are recomputed in-kernel: a few multiply-adds per Gaussian from references already in registers, plus a residual lookup. Because derived attributes are never allocated, each avatar's per-frame materialized size depends only on its Gaussian count and residual budget. This makes allocation sizes regular enough for an STAlloc-style plan over a small set of size classes inside vAttention-style reserved regions.
Baselines
The naive reference point is the unmodified avatar pipeline. It materializes all attributes in fp32 arrays through PyTorch's default caching allocator, with avatars added and removed as the workload dictates. It shows how much the problem exists at all.
The strongest quality-and-efficiency competitor is GoMAvatar itself, run with its released per-subject models on ZJU-MoCap. It reports 30.37 dB PSNR, 43 FPS on an A100, and 3.63 MB per subject (GoMAvatar). It is the strongest because it already exploits one hand-chosen cross-attribute dependency and is compact. We run it at identical resolution, pose sequences and GPUs, in both single-avatar and multi-avatar settings. Beating it means matching PSNR within 0.2 dB and LPIPS within noise while reading fewer bytes per avatar-frame and giving lower 99th-percentile frame time at 32 or more avatars. No reported result found on Lacuna for multi-avatar frame time or fragmentation in avatar rendering, so all multi-avatar numbers are our own measurements.
The strongest compression competitor is per-attribute ("vertical") compression. It combines fp16 plus per-attribute vector quantization in the style of Mobile-GS, which reached 4.6 MB from about 840 MB on static scenes with near-original PSNR. We give it the same byte budget and the same tuning sweep as our method, and it also stacks with ours. Beating it means lower bytes at equal quality, or equal bytes at better quality.
The strongest memory competitor is a static per-avatar pool. Each avatar gets a preallocated maximum-size buffer inside a reserved virtual range, backed by 2 MB pages. Beating it means lower total reserved memory at equal frame time under the churn workload.
Three ablations isolate the key ingredient. The first uses the size-only dependency graph ($\lambda = 0$, i.e., Virtual's objective) with our layout. The second uses our layout-aware graph with a default allocator. The third uses no virtualization but the same allocation plan. If the size-only graph matches the layout-aware one on frame time, the coupling claim is wrong.
Why not just…?
Why not just run the pose network inside the rasterizer and materialize nothing? This is a real threat. GoMAvatar's shading-from-normals is already a small step in that direction. Fully fusing a pose-conditioned network per Gaussian per frame trades memory for compute, and on a large GPU with few avatars it may win. Our method sits at a middle point: cheap linear derivation for the predictable attributes, stored residuals for the rest. We therefore include a fused-network variant as an extra arm and report the crossover in avatar count and GPU class. If the fused variant dominates everywhere, including on a Jetson Orin, the method is not worth pursuing.
Why not just quantize each attribute independently? Tabular evidence says per-column schemes plateau. Virtual gains up to 40% over Parquet, and ICGS-Quantizer's joint coding beats per-attribute C3DGS and CompGS on Google Scanned Objects. But none of that evidence comes from articulated avatars, so the equal-budget comparison above settles it. Since the two approaches compose, the honest claim is additive savings, not replacement.
Why not just preallocate a fixed pool per avatar and turn on huge pages? For a fixed avatar population with fixed Gaussian counts, this probably solves fragmentation outright, and we do not claim otherwise. The pool pays in over-reservation when avatars differ in size or switch level of detail. The churn workload measures exactly that. TEMERAIRE's finding that default huge-page mechanisms lose coverage under churn suggests the pool alone is not free. If the pool matches us on reserved memory and tail latency, we will report that the memory contribution reduces to co-location of reference and derived attributes, and nothing more.
Proposed Experiments
Quality is measured on ZJU-MoCap and People-Snapshot with PSNR, SSIM and LPIPS on novel poses and views. THUman4.0 serves as the loose-clothing stress test. Systems metrics are bytes read per avatar-frame, median and 99th-percentile frame time, fragmentation ratio (reserved but unusable memory over reserved memory), and total reserved memory. We run on an A100 and on a Jetson AGX Orin, the edge platform MemGS used. The churn workload replays traces in which 8 to 64 avatars built from these subjects join, leave and change level of detail at scripted rates.
The first decisive check comes before any method work. On the naive pipeline, we measure whether address translation matters on the GPU by comparing 64 KB and 2 MB backing pages at fixed layout, using Nsight counters where available. If page size changes frame time by under 2%, the translation motivation is dropped and the memory claim narrows to bytes and fragmentation. The second decisive check is the $\lambda$ sweep. The layout-aware graph must beat the size-only graph on 99th-percentile frame time, and must avoid Corra-style fetch penalties, at no more than 5% extra materialized bytes. The third check is quality: derived attributes must stay within 0.2 dB PSNR of the source avatar on held-out poses, reported separately for tight and loose clothing regions.
References
- Corra: Correlation-Aware Column Compression
- Lightweight Correlation-Aware Table Compression
- Virtual: Compressing Data Lake Files
- Image-Conditioned 3-D Gaussian Splat Quantization for Scene Archival
- MEGA: Memory-Efficient 4D Gaussian Splatting for Dynamic Scenes
- GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians
- Deformable 3D Gaussian Splatting for Animatable Human Avatars
- GoMAvatar: Efficient Animatable Human Modeling from Monocular Video Using Gaussians-on-Mesh
- Reducing GPU Memory Fragmentation via Spatio-Temporal Planning for Efficient Large-Scale Model Training
- vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention
- Beyond malloc efficiency to fleet efficiency: a hugepage-aware memory allocator
- Dissecting Scale-Out Applications Performance on Diverse TLB Designs (S)
- Towards Faster Reasoners by Using Transparent Huge Pages
- Mobile-GS: Real-time Gaussian Splatting for Mobile Devices
- MemGS: Memory-Efficient Gaussian Splatting for Real-Time SLAM