Code-generating language-model agents can now write fast GPU programs ("kernels") by running candidate code, reading hardware profiler output, and revising. They still decide whether a faster kernel is acceptable the same way unit tests do: its output must match a reference implementation to within a small numerical tolerance on a few fixed inputs. For neural rendering of animated human avatars, that test is both too strict and too weak. It is too strict because the avatar is judged by how the image looks, so large speedups from lower-precision math or dropping near-invisible primitives are fair game. It is too weak because the inputs that matter are body poses, and an approximation that looks perfect on a standing subject can fail visibly when the arms cross or the subject crouches. Those are exactly the poses where pose-dependent appearance networks are already fragile.
This note proposes a kernel-optimization agent for animatable Gaussian-splat avatars whose acceptance test is a pose search rather than a fixed test set. For each candidate kernel, a verifier searches the space of plausible body poses and camera views for the configuration that most increases perceptual error against the reference renderer. The candidate is accepted only if even that worst case stays under a visual tolerance. The agent writes in Triton, a Python-embedded GPU language that compiles to both NVIDIA and AMD hardware. That choice lets us ask a second question: whether the set of safe approximations differs across hardware. The claim is not that agents can fuse avatar pipelines. That would be an incremental port of existing agents to a new workload. The claim is that correctness for approximate graphics kernels should be defined over a distribution of poses, and that searching for bad poses catches regressions that random held-out poses miss.
Core Claim
The pipelines we target are GaussianAvatar and 3DGS-Avatar. GaussianAvatar decodes per-Gaussian offsets from a pose-conditioned U-Net. 3DGS-Avatar uses a non-rigid deformation network, learned skinning, and a small color network conditioned on pose. Both evaluate a network whose inputs are joint angles $\theta$ and then rasterize. These networks are trained on a narrow band of poses, and 3DGS-Avatar needs an explicit isometry regularizer to keep geometry from collapsing on out-of-distribution poses. We therefore hypothesize that numerical perturbations introduced by kernel approximations are amplified unevenly across pose space. Precision reductions, truncated view-dependent color terms, and relaxed opacity culling should do the most damage in hands, contact regions and garment folds. Known weak spots of these models, including the extremity problem and color leakage between touching limbs reported by Dream, Lift, Animate, predict where to look. If the hypothesis holds, verification on a random sample of test poses will accept kernels that fail on a non-trivial fraction of real motions, and an adversarial pose search will reject them at modest cost. The acceptance criterion is
$$\max_{\theta \in \Theta,\, v \in V} \; \mathrm{LPIPS}\big(R_{\text{cand}}(\theta, v),\, R_{\text{ref}}(\theta, v)\big) \le \tau,$$
where $\theta$ ranges over the set $\Theta$ of joint-limit-respecting poses near the capture's motion manifold, $v$ over camera views $V$, $R_{\text{cand}}$ and $R_{\text{ref}}$ are the candidate and reference renderers, and $\tau$ is a perceptual tolerance. We report results at several values of $\tau$ rather than fixing one.
Research Program
We first build a workload suite from the two avatar pipelines, split into stages. The stages are the pose-conditioned decoder, forward linear blend skinning of Gaussians, projection with tile binning and sorting, and alpha compositing. Neither KernelBench, which covers deep-learning operators and full vision and language models, nor TritonBench, which covers mostly LLM-serving operators, contains any of these stages. The suite is a necessary means here, not the contribution.
The optimizer reuses established machinery. It runs a coder-judge loop with profiler feedback in the style of CudaForge and TritonForge. It is seeded with a small per-architecture taxonomy of optimization strategies, following Hawkeye. Hawkeye's finding that about ten expert unit tests beat raw documentation is the reason we do not feed hardware manuals to the agent. Its report that FP4 kernels need "quantization tolerance tuning" is the motivating evidence that tolerance is a real design axis. To this loop we add an explicit menu of approximation moves: lower precision in the decoder and skinning, fewer view-dependent color coefficients, tighter opacity and footprint culling, and approximate depth ordering within a tile.
The new component is the verifier. It starts from the capture's training and test motions and runs a gradient-free search, using CMA-ES, over pose offsets constrained by joint limits and by distance to the observed motion set. It also samples camera distances, because zoom changes how many Gaussians fall into each screen tile. The search is warm-started from body regions flagged by the reference model's own reconstruction error. It returns the worst pose and view found, and those are fed back to the agent as a concrete counterexample image. AutoTriton and KernelBench instead check equality on fixed inputs. Our verifier differs in two ways: its test distribution is structured by the avatar model, and it returns a counterexample the agent can act on.
Baselines
The naive reference point is the original PyTorch implementation of each avatar pipeline with its stock CUDA Gaussian rasterizer, run unmodified. 3DGS-Avatar reports 50+ FPS for this configuration and GaussianAvatar 35 FPS. The second baseline is torch.compile in max-autotune mode on the non-rasterizer stages. Hawkeye reports agent kernels beating it by a 1.13x geometric mean in BF16 and 1.28x in low precision on standard workloads. No reported result was found on Lacuna for either system on avatar pipelines.
The strongest existing method is the same profiling-guided agent under a conventional acceptance test: fixed test inputs with an absolute-tolerance check, as in TritonForge. TritonForge reports a 42.7% optimization success rate and a 1.76x average speedup on successful TritonBench kernels. Every agent condition gets the same base model, the same eight-iteration cap TritonForge uses, the same dollar budget per stage, and the same hardware.
The key ablation keeps our approximation menu and perceptual tolerance but replaces the adversarial search with random verification. That verifier draws the same number of renders the search would spend, uniformly from held-out test poses and views. This isolates the pose search itself. Beating the baselines means two things. At matched LPIPS tolerance, we must accept kernels with higher end-to-end frame rate than the strict-equality agent. Relative to the random-verification ablation, we must have a significantly lower escaped-regression rate at no more than a small loss in speedup. An escaped regression is a kernel that passes verification but exceeds $\tau$ on a large audit set of unseen motions.
Why not just…?
Why not just validate on random held-out poses? This is the most serious threat, and the ablation above is designed to settle it. If escaped-regression rates are statistically indistinguishable between random and adversarial verification at equal render budgets, the pose-adversarial claim is false. The honest conclusion would then be that perceptual tolerance alone is the useful ingredient. We would still report the speed-quality frontier, but the paper would be about tolerance rather than search.
Why not just keep strict numerical equality? It is safe, but it rules out the approximations that matter most. We measure this directly as the speed gap between the strict-tolerance agent and our agent at small $\tau$. If that gap is small, approximation buys little on these workloads, and the proposal reduces to ordinary kernel fusion.
Why not just use torch.compile plus the stock rasterizer? The compiler cannot synthesize sorting, tile binning or compositing kernels, and it never trades accuracy for speed. It remains a required baseline, and on hardware where it closes most of the gap we will say so.
Why not just prune Gaussians offline and cache canonical features? Offline pruning is itself an approximation. It is validated on training poses and therefore faces exactly the verification question we study, so we include it as one move on the agent's menu. Pure caching with rigid skinning is known to lose pose-dependent wrinkles and to regress toward a mean appearance, as documented by Golyanik et al. and Neural Actor.
Proposed Experiments
We train avatars on PeopleSnapshot and six ZJU-MoCap sequences, following 3DGS-Avatar's splits. We add ActorsHQ for the Dream, Lift, Animate decoder, where the paper reports a reference quality of PSNR 25.58 and SSIM 0.9279 for novel-view synthesis. For ZJU-MoCap, the file gives only a qualitative claim that 3DGS-Avatar is comparable or better, with no number.
The audit set pairs each subject with motions from other ZJU-MoCap subjects plus a held-out set of extreme poses. The adversarial search never sees it. We run on an NVIDIA A100 and an H100 and on an AMD MI-series GPU, all through Triton.
We report four metrics. The first is end-to-end frame latency and per-stage speedup over both the naive and compiled baselines. The second is the change in PSNR, SSIM and LPIPS relative to the reference renderer on standard test poses. The third is the escaped-regression rate on the audit set. The fourth is verifier cost in renders and wall-clock time.
The decisive checks are these. First, does adversarial search lower escaped regressions relative to random verification at equal budget? Second, do the worst poses it finds concentrate in hands, contacts and folds, as the hypothesis predicts? Third, does the set of accepted approximations differ between NVIDIA and AMD targets at the same $\tau$? If they do, a verifier tuned on one vendor's hardware cannot be trusted on another. That would be a direct hardware-portability finding, extending the fragility across hardware generations that TritonBench reports for speed to visual correctness.
References
- GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians
- 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting
- Dream, Lift, Animate: From Single Images to Animatable Gaussian Avatars
- KernelBench: Can LLMs Write Efficient GPU Kernels?
- TritonBench: Benchmarking Large Language Model Capabilities for Generating Triton Operators
- CudaForge: An Agent Framework with Hardware Feedback for CUDA Kernel Optimization
- TritonForge: Profiling-Guided Framework for Automated Triton Kernel Optimization
- Hawkeye: Hardware-Aware GPU Kernel Optimization with Minimal Supervision
- AUTOTRITON: Automatic Triton Programming with Reinforcement Learning in LLMs
- VINECS: Video-based Neural Character Skinning
- Neural actor: neural free-view synthesis of human actors with pose control