Phase-Retrieval Supports for Sparse-View Gaussian Splatting of Wheat Canopies

Measuring plant traits such as organ length, width and volume by hand is slow and error-prone. 3D reconstruction from a ring of cameras can replace calipers, but the methods that work today need many cameras. When a cheap field rig supplies only three or four views, the reconstruction model can match the photos while inventing geometry that does not exist: semi-transparent "floaters" hang in front of the cameras, and occluded organs swell or vanish. Physicists solved a structurally similar problem decades ago. In phase retrieval, the task of recovering an object from diffraction images that record brightness but lose phase, solvers alternate between two steps. One fits the measurements; the other forces the estimate back inside a known region of space called the support. Crucially, that region is re-estimated as the solve proceeds, and points outside it are pushed out gently rather than deleted.

We propose to bring this step into sparse-view 3D Gaussian Splatting, which represents a scene as millions of small coloured, semi-transparent ellipsoids. The support would be built from per-organ segmentation masks and tightened during optimization. Gaussians outside it would have their opacity reduced through the softened "hybrid input-output" feedback rule from phase retrieval, not through hard deletion. The claim is concrete and falsifiable. On wheat plots seen from three or four cameras, this soft, adaptive support should recover organ traits, and volume above all, much closer to a 12-camera reconstruction than either hard visual-hull carving or depth-prior regularization does. It should also produce a smaller model that renders faster.

Core Claim and Why It Is Not Just Carving

The obvious version of this idea is weak, and we reject it at the outset. That version takes segmentation masks from each view, intersects their viewing cones into a visual hull, and deletes every Gaussian outside the hull. This is classical space carving. With three views the hull of a dense canopy is loose, so most floaters sit inside it. Hard deletion also has a documented failure mode under sparse views. Few-Shot Novel View Synthesis Using Depth Aware 3D Gaussian Splatting improves sparse-view Gaussian Splatting partly by disabling opacity-based pruning, because pruning collapses the sparse point cloud. Phase retrieval saw the same failure long ago, as summarized in The Numerics of Phase Retrieval. The hard-projection "error reduction" algorithm stagnates in local minima. Fienup's hybrid input-output (HIO) rule escapes them by applying negative feedback outside the support instead of zeroing it. The support is also re-estimated by "shrinkwrapping", which repeatedly blurs and thresholds the current estimate, as in the single-particle imaging work Reconstruction algorithm for single-particle diffraction imaging experiments.

What we borrow is therefore not "a support exists." We borrow two specific mechanisms. The first is a support that starts as a loose per-organ mask hull and tightens as the reconstruction improves. The second is a soft feedback update on opacity outside that support. Let $o_i^k$ be the opacity of Gaussian $i$ at iteration $k$, let $o_i'$ be its opacity after a photometric gradient step on the training views, let $S_k$ be the current support region, let $\mu_i$ be the Gaussian's centre, and let $\beta \in (0,1)$ be a feedback strength. The update is

$$ o_i^{k+1} = \begin{cases} o_i' & \mu_i \in S_k \\ \max\left(0,\; o_i^k - \beta\, o_i'\right) & \mu_i \notin S_k \end{cases} $$

Opacity outside the support decays at a rate tied to how strongly the photometric loss is trying to keep it. A Gaussian is not killed in one step, so densification inside the support has time to take over the role the floater was playing. Our central bet is that this outperforms hard carving in the sparse regime. The key ablation tests exactly that.

The role of each prior paper is as follows. Wheat3DGS motivates the work and supplies the pipeline. It reconstructs field wheat plots from a 12-camera rig with Gaussian Splatting, links SAM instance masks across views by 3D overlap, and measures head length, width and volume. Its volume error of 40.2% mean absolute percentage error (MAPE), set against 15.1% for length, shows that volume is the weakest trait even with dense views. Its traits come from a convex hull applied after reconstruction, not from constraints applied during it. We reuse its mask-association machinery to define per-instance supports, then remove most of its cameras. The phase-retrieval papers supply the method. The sparse-view papers (RegNeRF, Depth-Aware 3DGS, MVSplat) are the competitors, and their stated weaknesses shape the design. RegNeRF's depth smoothness erases thin structures such as awns. Monocular depth breaks at foliage discontinuities. MVSplat's cost volume assumes matchable, non-repetitive texture.

We drop two analogies the research file floats, because they do not survive scrutiny. Incoherent RGB images have no meaningful reciprocal-space oversampling, and "coded diffraction masks" for cost volumes have no clear physical counterpart. Only the support and feedback machinery transfers cleanly.

Research Program

The system has three parts. The first builds a per-instance support. Wheat heads are detected and segmented in each of the $V$ sparse views (YOLO plus SAM, as in Wheat3DGS). Instances are associated across views using the iterative 3D-overlap matching from Wheat3DGS, and each instance's support starts as the intersection of its mask cones, dilated by a small margin to absorb mask error. Background and stem regions receive a coarse scene-level support, so the method does not silently discard non-organ geometry.

The second part is shrinkwrapping. Every $T$ iterations, each instance support is replaced by its intersection with a thresholded, Gaussian-blurred density of that instance's current Gaussians. The blur width shrinks over time, following the shrinkwrap schedule in coherent diffraction imaging. The mask-cone hull is kept as an outer bound, so the support can only tighten.

The third part is the feedback update above, run inside a standard Gaussian Splatting optimizer. Densification is restricted to the support, and we use an amplitude-style photometric loss. The Numerics of Phase Retrieval reports that losses on amplitudes are more stable than losses on squared intensities. In RGB the analogue is an L1 loss rather than an L2 loss on pixel values, which is already common practice, so we treat it as a default rather than a contribution.

Latency enters as a constraint and as a measured benefit. The support test costs one point-in-region check per Gaussian per projection step, which is negligible next to rasterization. Because floaters are removed, we expect fewer Gaussians, which lowers memory and raises frame rate. Memory is the bottleneck on edge devices noted in Compression in 3D Gaussian Splatting. We will report wall-clock reconstruction time, Gaussian count and rendering frames per second, in addition to accuracy.

Baselines

The naive reference point is vanilla Gaussian Splatting trained on the same three or four views, with standard pruning and densification. It shows how badly the sparse regime fails on canopies, and every method must clear it. The upper reference is Wheat3DGS with all 12 cameras. It reports SSIM 0.843 and length, width and volume MAPE of 15.1%, 18.3% and 40.2% against laser-scan ground truth on the ETH Zurich wheat plots (Wheat3DGS). We do not expect to beat the 12-view numbers. Success means closing most of the gap between sparse vanilla splatting and this reference.

The strongest per-scene competitor is Depth-Aware 3DGS (Few-Shot Novel View Synthesis Using Depth Aware 3D Gaussian Splatting). It adds a scale-invariant monocular depth loss, keeps all Gaussians, and lowers colour complexity, and it reports a 10.5% average PSNR gain over vanilla splatting on LLFF. It is the most directly comparable method, because it is also a regularizer on per-scene Gaussian optimization. The strongest feed-forward competitor is MVSplat (MVSplat). It predicts Gaussians from a plane-sweep cost volume and reports PSNR 26.39 on RealEstate10K at about 22 frames per second. We include it because it wins outright on speed. We also run RegNeRF, which reports PSNR 18.89 on 3-view DTU, as the classical smoothness regularizer. We expect it to erase awns. No reported result was found on Lacuna for any sparse-view method on the wheat or maize phenotyping datasets, so all sparse-view numbers on those datasets will be produced by us.

Fairness rules apply to all methods. Every method receives the same view subsets, the same calibrated poses (taken from the full rig, so pose failure does not confound the comparison), the same masks where masks are used, and the same iteration budget and GPU. Each method gets an equal hyperparameter search of roughly 20 trials, tuned on held-out plots. MVSplat is evaluated both off the shelf and fine-tuned on the training plots.

Three ablations isolate the key ingredient. The first is hard carving to the fixed mask hull, i.e. space carving. The second is hard carving to the shrinkwrapped support, which tests whether adaptivity alone suffices. The third is soft HIO feedback with a fixed hull, which tests whether feedback alone suffices. The proposal counts as beating the field if it reduces volume MAPE below both Depth-Aware 3DGS and the best hard-carving ablation by a margin larger than the plot-to-plot standard deviation, without losing held-out-view SSIM.

Why not just…?

Why not just carve with the visual hull from the masks? This is the most serious threat, and it is the first ablation. With three views the hull of a dense plot is loose, and the evidence from Depth-Aware 3DGS that hard pruning collapses sparse reconstructions suggests that hard carving will either leave floaters inside the hull or starve real geometry. If hard carving to the shrinkwrapped support matches the soft feedback rule, then the phase-retrieval feedback adds nothing. In that case we would report that the contribution reduces to adaptive per-organ supports, a smaller but still useful result.

Why not just add a monocular depth prior? Depth-Aware 3DGS does exactly this and is a strong baseline. Its gains, however, were shown on LLFF scenes, and a monocular network must guess depth at every leaf-over-leaf discontinuity in a canopy. We will measure this directly by comparing the monocular depth maps against laser-scan depth on the wheat plots. We also try combining the depth loss with our support. If the combination wins, the two methods are complementary rather than competing.

Why not just use a feed-forward model such as MVSplat, or a generative view prior such as CAT3D? MVSplat is far faster, and if it matches trait accuracy after fine-tuning, our per-scene method loses its reason to exist for throughput. That comparison is in the experiments. Its cost volume, however, relies on cross-view matching that self-similar wheat heads confuse. CAT3D fills in unseen views with a multi-view diffusion model and reports PSNR 16.62 on 3-view Mip-NeRF 360, but views it invents are a liability when the output is a genotype measurement. We include CAT3D only as a reference on image quality and do not count it as a trait method.

Why not just keep the 12-camera rig, or use 2D detection? The 12-camera result is our ceiling, but rig cost and field throughput scale with camera count, and that is the bottleneck we target. 2D detectors count heads on the Global Wheat Head Detection dataset with [email protected] of 0.77 and relative counting error of 39%, but they cannot measure volume at all.

Proposed Experiments

The primary testbed is the ETH Zurich wheat plot dataset from Wheat3DGS: 7 plots, 12 cameras, with laser-scan ground truth. If it cannot be obtained, we will build a comparable rig. For each plot we draw multiple random and evenly spaced subsets of 2, 3, 4 and 6 cameras, and the remaining cameras serve as held-out views. Metrics are SSIM, PSNR and LPIPS on held-out views. Trait accuracy is length, width and volume MAPE against laser scans, using Wheat3DGS's trait extraction unchanged. We also report Gaussian count, reconstruction time and render speed.

The second testbed probes slender structures. It is the indoor maize seedling dataset from Auto3DPheno, which reports plant height relative error of 0.76% and stem width error of 2.93% from dense video. We subsample frames to 3 to 6 views and measure height, stem width and leaf length against the caliper ground truth. This tests whether shrinkwrapping erodes thin stems, which is the risk opposite to floaters. A generality check on 3-view DTU and 3-view LLFF, where the RegNeRF and Depth-Aware 3DGS figures are reported, will show whether the method helps beyond plants or only where good instance masks exist.

Three results decide the project. First, at 3 views on the wheat plots, soft feedback with shrinkwrapping must beat hard carving to the same support on volume MAPE. Without that, the phase-retrieval part has no case. Second, the gain must grow as views decrease, since a support constraint should matter most when the photos constrain least. Third, on maize, stem width error must not rise relative to vanilla splatting, which would reveal over-shrinking. We will also report failure cases where segmentation misses an occluded head, since a support built from masks cannot recover an organ that no mask sees. That limitation is also the clearest pointer to future work.