QuantileGS: Quantile-Based Conformal Bounds on RGB Rendering Errors for Novel View Synthesis with 3D Gaussian Splatting

Haiyang Xu, Ziye Zhang, Bin Pan, Xia Xu, Zhenwei ShiarXiv 2026 · arXiv

TL;DR

QuantileGS is a post-hoc framework that adds statistically rigorous uncertainty bounds to 3D Gaussian Splatting (3DGS) renderings. By combining learned error quantiles with conformal calibration, it provides pixel-wise "error bars" that accurately predict rendering discrepancies at unseen viewpoints, significantly improving reliability for downstream tasks like SLAM and active view selection.

What Problem Does This Paper Address?

3D Gaussian Splatting (3DGS) has revolutionized novel view synthesis, allowing us to render photorealistic scenes from just a few photos. However, when we move the virtual camera to a viewpoint it hasn't seen before, the model might "hallucinate" or produce errors due to occlusions or limited data.

The big problem is reliability: how do we know which pixels to trust? Existing uncertainty quantification (UQ) methods might tell you where the model is "confused," but they don't provide a concrete guarantee. They can't say, "The error at this pixel is less than $X$ with 90% probability." Without these reliable bounds, autonomous systems using 3DGS for navigation or mapping might make overconfident, catastrophic decisions.

Methods

QuantileGS introduces a three-phase "post-hoc" framework, meaning it can be applied to a 3DGS model that has already been trained.

  1. Rendering-Error Computation: The authors take a frozen, pretrained 3DGS model and measure the actual Mean Absolute Error (MAE) on a set of "fitting views" where the ground truth is known.
  2. Ordered Gaussian Quantile Fields: Instead of retraining the whole scene, they add two new scalar attributes to every Gaussian: a lower and an upper error quantile. These are rendered using the same alpha-blending process as the colors. This creates a "quantile field" that predicts the range of potential errors for any given viewpoint.
  3. Scale-Normalized Conformal Calibration: This is the "secret sauce." Using a separate set of held-out calibration images, the framework adjusts the predicted upper bounds. It uses the "quantile gap" (the difference between the upper and lower predicted quantiles) to scale the correction spatially. This ensures that the final bounds provide finite-sample marginal coverage—a statistical guarantee that the true error will fall within the predicted bound a specified percentage of the time (e.g., 90%).
Figure 2. QuantileGS overview. Phase 1 computes RGB-MAE on Vfit using a frozen pretrained 3DGS model. Phase 2 learns ordered Gaussian error quantiles with fixed blending weights and background contributions. Phase 3 calibrates the upper prediction on Vcal, using the interquantile gap with a positive floor as the local scale. The prediction procedure renders the learned quantiles for a new view and applies a locally scaled conformal correction to produce an RGB-MAE upper-bound map.

Datasets

The authors evaluated QuantileGS on several standard benchmarks for novel view synthesis and robotics:

  • LF (Light Field): A dataset with dense viewpoint sampling.
  • LLFF (Local Light Field Fusion): Forward-facing scenes.
  • Mip-NeRF 360: Complex, unbounded 360-degree scenes.
  • Replica: Used specifically for evaluating the method's impact on RGB-D SLAM (Simultaneous Localization and Mapping).

Key Insights and Ideas

  • Preserving the Renderer: By learning quantiles as Gaussian attributes and using the original blending weights, the authors ensure the uncertainty maps are perfectly aligned with the geometry of the scene without changing the original (often high-quality) RGB output.
  • Spatially Adaptive Bounds: Unlike simple global calibration, the "scale-normalized" approach means the error bars are tighter in well-observed areas and wider in regions with complex geometry or occlusions.
  • From Heuristics to Guarantees: Most UQ methods provide "scores" that correlate with error. QuantileGS uses Conformal Prediction to turn these heuristic scores into rigorous, calibrated bounds that meet a user-defined reliability target.

Key Results

The quantitative improvements are striking, particularly in how well the predicted bounds match reality:

  • Calibration Works: On the LLFF dataset, uncalibrated bounds only covered the true error 69.6% of the time when aiming for 90%. After QuantileGS calibration, the empirical coverage hit 89.6%.
  • Better Error Ranking: QuantileGS achieved lower AUSE (Area Under Sparsification Error) and higher Pearson correlation with actual errors than established baselines like Variational-3DGS and FisherRF.
  • Downstream Impact:
    • Active View Selection: Using these bounds to pick the "next best" camera view improved reconstruction quality (PSNR) by 1.729 dB over random selection.
  • SLAM: In robotic mapping tasks on the Replica dataset, QuantileGS reduced translational Relative Pose Error (RPE) by approximately 91% compared to the VarSplat baseline.
Figure 3. Qualitative comparison of Variational-3DGS [28], FisherRF [23], PRIMU [17], Manifold [30], and QuantileGS-C (Ours). Each scene shows uncertainty maps. Scenes from top to bottom: Mip-NeRF 360 bicycle and counter, LLFF fern and horns, and LF basket.

Limitations and Future Work

  • Marginal vs. Conditional Coverage: The statistical guarantee is "marginal," meaning it holds on average across all pixels and views. It doesn't strictly guarantee 90% coverage for every individual pixel or every specific "difficult" view.
  • Exchangeability: Conformal prediction relies on the assumption that calibration and test data are "exchangeable." Strong view-dependent effects (like sharp specularities) might violate this, though the authors' scale-normalization helps mitigate it.
  • Computational Cost: While post-hoc, rendering two additional fields and performing calibration adds some overhead to the inference pipeline.

Future work could explore tighter "conditional" guarantees or integrating these bounds more deeply into the initial 3DGS training process rather than keeping the model frozen.

Summary updated

Abstract

Uncertainty quantification for novel view synthesis with 3D Gaussian Splatting (3DGS) assesses rendering reliability at unseen viewpoints. This reliability depends on the discrepancy between rendered and true colors, which cannot be measured without ground-truth images. We therefore quantify uncertainty in this unknown discrepancy through upper bounds on per-pixel RGB mean absolute error. However, learned bounds may underestimate these errors and overstate rendering reliability, motivating calibration to a prescribed coverage level. In this paper, we propose QuantileGS, a post-hoc framework that combines learned rendering-error quantiles with scale-normalized conformal calibration to produce spatially adaptive error bounds. The framework comprises three phases. Phase 1, Rendering-Error Computation, measures RGB mean absolute error between observed images and predictions from a frozen 3DGS model. Phase 2, Ordered Gaussian Quantile Fields, learns two scalar attributes per Gaussian to predict lower and upper rendering-error quantiles. Phase 3, Scale-Normalized Conformal Calibration, calibrates the upper prediction on held-out images, using the quantile gap to scale the correction. Under the stated assumptions, the resulting bounds provide finite-sample marginal coverage. At a nominal coverage level of 90%, calibration raises mean empirical test coverage from 69.6% to 89.6% on LLFF and from 80.1% to 92.1% on Mip-NeRF 360. On LF, coverage remains conservative at 96.3% before and 96.4% after calibration. Applications to active view selection, Gaussian pruning, and RGB-D simultaneous localization and mapping illustrate how these bounds inform downstream decisions.
BibTeX
@misc{xu2026quantilegs,
  author = {Haiyang Xu and Ziye Zhang and Bin Pan and Xia Xu and Zhenwei Shi},
  title = {{QuantileGS: Quantile-Based Conformal Bounds on RGB Rendering Errors for Novel View Synthesis with 3D Gaussian Splatting}},
  year = {2026},
  eprint = {2611.21253},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21253},
}
Figure