Parallax as Evidence: Feed-Forward 3D Entity Proposals for Hallucination-Resistant Questions about Camouflaged Objects

Vision-language models, which are large language models fed with image features, often answer questions about objects they cannot actually see. They fall back on what usually co-occurs with the scene: a forest suggests a deer, and a kitchen suggests a sink. Camouflaged objects push this failure to its extreme. When an animal's color and texture match its background, the image carries almost no 2D signal. The model then either misses the object or "finds" one because the context makes it likely, and it may be right for the wrong reason. Our proposal rests on an old fact from vision. Camouflage defeats appearance, but it does not defeat parallax. When the camera moves, a camouflaged object at a different depth shifts against its background, however well its texture matches.

We propose to turn that parallax into explicit evidence for a vision-language model. We do this within a strict latency budget and without optimizing a 3D model for each scene. The pipeline starts from a few posed frames. A feed-forward Gaussian-splatting reconstructor, a network that predicts a 3D scene made of small colored blobs in one forward pass, produces multi-view depth in under a second. From this depth we extract "geometric entity proposals": regions bounded by depth jumps that no color edge explains. These proposals are handed to the language model as extra object-level views. The model's existence claims are then checked against them. The claim is that multi-view geometry, and specifically not a single-image depth guess, is the ingredient that reduces both missed objects and prior-driven hallucination in low-contrast scenes.

Core Claim

In multi-view or short-video captures of camouflaged scenes, entity evidence derived from multi-view stereo parallax lowers object-hallucination error in vision-language models. We measure this error in two directions: false "yes" answers driven by context, and false "no" answers on present but hidden objects. We expect the gain to be substantially larger than what 2D test-time augmentation, 2D detector-based view trees, or monocular depth deliver. The overhead should be under one second per scene, and there should be no per-scene gradient descent.

The claim makes a sharp prediction. The advantage should appear only when foreground and background are texture-matched and parallax is available. It should shrink toward zero on single images and on high-contrast scenes. It should also be largest where a monocular depth network fails, because monocular depth networks infer depth from the same appearance cues that camouflage destroys.

Why This Intersection, and What Is Already Done

Each of the three directions supplies one piece. Low-contrast saliency work motivates the use of geometry. Fast, recurrent, attentional modulation improves saliency representation and scene recognition documents that center-surround contrast breaks down when targets resemble their surroundings. DPANet breaks such ties with depth, learning a "depth potentiality" score that gates how much depth to trust. DPANet also constrains us: it reports failures for small, distant, or multiple objects where depth lacks detail. We therefore borrow its gating idea and stratify our results by object size and distance. Camouflaged Object Detection supplies the task definition, the COD10K dataset, and the SINet detector. All of these are single-image, so parallax is never available in that setting.

The feed-forward reconstruction literature supplies the method and sets the latency constraint. DepthSplat predicts Gaussians in 0.6 seconds from 12 views of 512×960 pixels. Its separate multi-view stereo branch and monocular Depth Anything V2 branch make it a natural tool for our key ablation. GS-LRM reconstructs from 2–4 posed images in 0.23 seconds. ReSplat refines Gaussians recurrently without test-time backpropagation. All three still need calibrated poses and include no semantic component. We accept the pose requirement as a stated limitation, not something we solve here. The per-scene optimization loop described in Score Jacobian Chaining serves only as a geometry-quality upper bound that we refuse to pay for at deployment.

The hallucination literature supplies the failure we target and the decoding machinery. Evaluating Object Hallucination in Large Vision-Language Models shows that co-occurrence priors drive false "yes" answers and introduces POPE, a yes/no object-polling benchmark with random, popular, and adversarial negatives. Villa et al. warn that correct answers can come from priors rather than pixels. This matters especially for camouflage, where the prior is often right, and it is why we add counterfactual scenes.

The closest existing work is Game on Tree. It builds a scene/relation/entity tree from GroundingDINO and RAM detections and uses Shapley values to pick the right granularity for contrastive decoding. Its entity layer comes from 2D detectors, which are exactly the component that fails on camouflage. When the detector misses the target, the tree has no node for it, and decoding falls back on the language prior. We keep the idea of object-level views for decoding but replace the source of entities with geometry. Hallo3D runs the flow the other way: it uses a vision-language critic to fix 3D generation inside a slow per-scene loop. 3D-GRAND reduces hallucination with dense 3D grounding, but on synthetic CAD rooms with clean boundaries, and it requires retraining. CapSal lets captions guide saliency. We reverse the direction and let geometry guide language, which also avoids CapSal's weakness of hallucinated caption words steering the mask. To our reading of the file, no work feeds feed-forward multi-view geometry into vision-language decoding for low-contrast objects.

Research Program

The system has three stages, all forward-only.

The first stage is reconstruction. Given $N$ posed frames (2 to 12), we run DepthSplat and read out two depth maps per view: $D^{mv}$ from the multi-view stereo branch and $D^{mono}$ from the monocular branch.

The second stage produces geometric entity proposals. For each view we compute depth-discontinuity edges from the fused depth and suppress those that coincide with strong color edges. What remains are boundaries that geometry sees and appearance does not. These edges seed a grouping of the predicted Gaussians into connected components separated by depth jumps. Each component is rendered as a tight crop with a soft mask in the reference view. Each proposal also gets a reliability score. This score adapts DPANet's potentiality idea: it is trained to predict mask IoU against ground truth on our synthetic training split from features such as multi-view agreement, proposal size, and depth variance. Low-reliability proposals are dropped, so noisy depth does not inject false objects.

The third stage is grounded decoding. The vision-language model receives the original image plus the retained proposal crops, which slot into an entity layer in the style of Game on Tree. For yes/no existence questions, we apply contrastive decoding between the image-only input and the image-plus-proposals input. We also add a verification step. A "yes" is kept only if some proposal's crop, queried alone, supports the object name above a threshold. A "no" is revisited if a high-reliability proposal exists whose crop the model labels as the queried category. Nothing in the stack is trained except the small reliability scorer. The vision-language model and the reconstructor stay frozen.

Alongside the method, we build the evaluation set, because none in the file combines camouflage, multiple views, and hallucination polling. We call it Camo-POPE. Its synthetic part places Objaverse and GSO objects into DL3DV and RealEstate10K scenes, retextured with the local background texture, and renders them along the original camera paths. This gives known poses, exact masks, and a contrast dial ranging from matched texture to original texture. Crucially, every scene has a paired counterfactual with the object removed and everything else identical. A small real part consists of hand-captured posed clips of camouflaged objects, used only for testing. Questions follow POPE's random, popular, and adversarial negative sampling.

Baselines

The naive reference point is the frozen base model, LLaVA-1.5-7B, given the reference frame alone and decoding greedily. On standard POPE, RITUAL reports 84.13% accuracy for this model, and Hallucination Begins Where Saliency Drops reports F1 of 85.4. We will re-measure it on Camo-POPE, where we expect it to be much lower.

The strongest reported decoding-time competitor on POPE in the file is RITUAL. It decodes with the original image plus a randomly transformed copy and reaches 88.87% POPE accuracy on LLaVA-1.5. It is the strongest cheap alternative because it needs no extra model. We run it with the same base model, the same questions, and its published transform set. It gets a tuning budget on our validation split equal to the one spent on our thresholds. The second decoding competitor is SGRS plus LocoRE from Hallucination Begins Where Saliency Drops, which reports POPE F1 rising from 85.4 to 87.5 and CHAIR sentence-level hallucination falling from 48.0 to 35.6. We run it under the same conditions.

The most direct structural competitor is Game on Tree with its GroundingDINO/RAM entity layer. It is the method we modify, so it isolates the value of our entity source. The file reports no POPE result for it; it reports 13.1% and 21.0% reductions in sentence- and instance-level CHAIR relative to HALC, at 192.91 ms per token. We run it with the same tree construction and swap only the entity source.

Three ablations isolate the key ingredient. The monocular variant builds proposals from $D^{mono}$ alone, using Depth Anything V2 in the identical pipeline with the same reliability scorer retrained. The SINet variant builds proposals from a camouflage detector's 2D masks; SINet reports an S-measure of 0.771 on COD10K. The per-scene variant builds proposals from 3D Gaussian splatting optimized per scene, which is an upper bound on geometry quality at a large latency cost. An oracle-mask variant bounds how much any entity source could help.

Beating the field means one specific result. Our method must exceed RITUAL and the Game on Tree variant on Camo-POPE F1 in the texture-matched condition, with non-overlapping bootstrap intervals, and it must exceed the monocular variant there. Its overhead must remain under one second per scene, which DepthSplat's reported 0.6 seconds makes plausible.

Why not just…?

Why not just use RITUAL-style 2D augmentation? Flips, rotations, and color jitter move foreground and background together, so a texture-matched boundary stays invisible after the transform. RITUAL itself also doubles forward passes and reports diminishing returns. We do not assume this settles the question. The contrast dial in Camo-POPE directly tests whether RITUAL's gain disappears as contrast drops while ours persists. If RITUAL matches us in the matched-texture condition, our central claim fails.

Why not just run a monocular depth network? This is the real threat. Depth Anything V2 is cheap and needs one image. Our hypothesis is that monocular depth, being learned from appearance, partly inherits the camouflage. DPANet and DepthSplat both document that single-image or raw depth is unreliable in clutter and textureless regions. The monocular-variant ablation settles it. If monocular proposals match multi-view proposals on matched-texture scenes, the reconstruction step is unnecessary. We would then report that result and narrow the method to monocular depth.

Why not just give the vision-language model all the frames? Motion reveals camouflage to humans, so a video-capable model might pick up parallax implicitly. But Why LVLMs Are More Prone to Hallucinations in Longer Responses shows that these models drift away from visual tokens, and nothing in the file shows them recovering depth from parallax. We will test a multi-frame-input baseline with the same frames. This is the second place where the idea could fail honestly.

Why not just use LocoRE or rejection sampling on attention saliency? These methods stabilize the text stream but add no visual evidence. Hallucination Begins Where Saliency Drops itself reports confident, high-saliency hallucinations that slip past its thresholds. Prior-driven errors on camouflaged scenes are likely to be of this kind. The comparison on adversarial negatives will show whether this holds.

Decisive Checks

Four results decide the proposal. First, the gap between multi-view and monocular proposals, plotted against the contrast dial, should grow as contrast falls. Second, on paired counterfactual scenes, the rate at which answers correctly flip when the object is removed should rise under our method. This measures grounding rather than lucky priors, the concern raised by Villa et al. Third, on single-image COD10K, where parallax is unavailable, we predict no gain over the monocular variant. A gain there would mean something other than parallax is doing the work. Fourth, results stratified by object size and distance should show where DPANet's warning about small, distant objects bites, and whether the reliability gate prevents the method from harming accuracy there. We also report CHAIR on free-form captions, proposal-mask S-measure against SINet on our data, and added latency per scene.