Carved Backdoors: Jointly Optimizing Shape and Shading for Stealthy 3D Triggers

Modern 3D perception systems — autonomous-vehicle detectors that fuse camera images with laser scans, or classifiers that consume raw 3D point clouds — can be secretly corrupted during training. A backdoor attack plants a hidden rule in the model: inputs behave normally unless they carry a specific secret marker, at which point the model outputs whatever the attacker wants. The hard part is not planting the rule; it is making the marker survive the journey from a physical object, through multiple sensors and projection layers, into the model, while staying invisible to both human inspectors and automated filters. The literature shows each single-modality marker fails somewhere: plain image patches are torn apart when a fusion pipeline projects 2D pixels onto sparse 3D laser points, achieving attack success rates below 10% (BadFusion); pure coordinate perturbations get stripped by outlier filters unless pushed into exotic spectral domains (PointBA, SPBA); and hand-built physical decals need laborious material modeling to register on LiDAR at all (MOBA).

This proposal borrows the attacker's missing tool from an unrelated, benign literature: joint geometry-appearance optimization, where 3D shape and surface texture are refined together inside one differentiable rendering loop, so that each can compensate for and correct the other (OMeGa, Text2Mesh). The core claim is that a backdoor trigger carved jointly into an object's surface geometry and its surface shading — a shallow embossment plus a matching photometric pattern, co-optimized on the mesh manifold — can be simultaneously consistent across sensors, nearly invisible to geometric filters, and imperceptible to humans, in a way no single-modality or sequentially-designed trigger can.

Core Claim

Let $G$ be an object's mesh, $A$ its appearance (texture or Gaussian splats bound to faces), and $\Delta G$, $\Delta A$ the attacker's perturbations. We propose optimizing the pair $(\Delta G, \Delta A)$ jointly through a differentiable renderer:

$$ \min_{\Delta G, \Delta A} \; \mathcal{L}_{\text{backdoor}}\big(f(R(G+\Delta G,\, A+\Delta A))\big) \;+\; \lambda \,\mathrm{CD}(G+\Delta G,\, G) $$

where $R$ is a differentiable renderer (photometric and, where applicable, LiDAR-intensity simulation), $f$ is the victim model, and $\mathrm{CD}$ is Chamfer Distance between perturbed and clean geometry. The design goal is an asymmetric split of labor: keep $\Delta G$ small enough that the geometric footprint is below the threshold of Statistical Outlier Removal and curvature-anomaly defenses, and let the jointly optimized appearance $\Delta A$ carry the remaining activation in camera space — with the renderer guaranteeing that the two components stay mutually consistent across viewpoints, distances, and sensor projections. The binding of appearance primitives to mesh faces via barycentric coordinates, as in OMeGa, is what makes the trigger inherently multi-view and cross-modal consistent rather than heuristically placed. A stronger variant pushes this into an AND-trigger: neither $\Delta G$ alone nor $\Delta A$ alone activates the backdoor (each is individually innocuous, echoing how backdoors hide in dormant capacity that rarely fires on clean data, per Fine-Pruning); only their aligned combination does, making the trigger dramatically harder to discover by auditing either modality in isolation.

Research Program

Three components. First, the trigger optimizer: an extension of a joint mesh-plus-splat reconstruction pipeline (OMeGa-style barycentric binding, with its photometric $L_1$/LD-SSIM losses, Laplacian smoothing, and normal consistency) repurposed as an attack objective, with an explicit LiDAR forward model (Oren–Nayar-style reflectance and distance-dependent point density, following the sensor physics MOBA established) so the optimized trigger projects into plausible point returns rather than being shredded. Second, the poisoning protocol: a fraction of training scenes show the object carrying the co-optimized trigger, relabeled to the attacker's target, with the remainder clean — the standard poison/clean split under which PointBA needed only 5% poisoning for over 95% attack success. Third, the analysis: a characterization of the trade-off frontier between geometric footprint (Chamfer Distance), activation strength (attack success rate), and stealth (perceptibility of renders), testing the hypothesis that joint optimization dominates sequential design along this frontier. As a secondary defensive thread, we will test whether the converse holds — whether a defender can flag backdoors by measuring misalignment between surface shading gradients and physical curvature, since benign joint pipelines enforce exactly that alignment (Tactile DreamFusion).

Baselines

The strongest existing competitor is BadFusion (BadFusion), which attacks the same camera-LiDAR fusion detectors we target. It heuristically searches for high-density LiDAR-projection regions with a Faster R-CNN detector and places uniform solid-color 2D stickers there, reporting attack success rates of 95.28%, 90.54%, and 92.44% on MVX-Net, Focals Conv-F, and EPNet respectively, with clean mean Average Precision of 88.65% on MVX-Net. It will be run on the same KITTI splits, the same victim architectures, the same poisoning rate and tuning budget as our method; we beat it only if we match or exceed its attack success rate and clean mAP while additionally surviving the defenses BadFusion was never tested against (Statistical Outlier Removal, curvature anomaly detection) and while reducing trigger detectability under human inspection. The naive reference point is a standard 2D patch backdoor of the BadDet family, which BadFusion measured at below 10% attack success rate against fusion models — it establishes the floor. On the point-cloud side, SPBA (SPBA) is the strongest appearance-blind competitor: it hides spectral perturbations in high-curvature regions, reporting 97.58% average attack success rate on ModelNet40 with the lowest average Chamfer Distance of 1.08; we compare against it on ModelNet40 under the same defenses, expecting it to collapse when camera input is added to the pipeline while our trigger does not. Finally, the decisive ablation is the decoupled variant of our own method: fix the geometry perturbation first, then optimize texture onto it sequentially. If joint optimization matters, this ablation should show measurably worse multi-view consistency or higher detectability at equal attack success rate; OMeGa's 47.3% Chamfer-$L_1$ reduction over decoupled 2DGS baselines in benign reconstruction is the motivating evidence that the coupling, not the components, does the work.

Why not just…?

Why not just use 2D image-space triggers on the camera inputs? This is measured, not conjectured: BadFusion directly evaluated traditional 2D backdoors against fusion detectors and found attack success rates below 10%, because the 2D-to-3D projection shreds pixels that do not correspond to dense laser returns. BadFusion itself recovers high success only through heuristic placement in dense regions — a workaround our renderer-bound trigger replaces with a principled, multi-view-consistent design.

Why not just perturb 3D coordinates, as PointBA and SPBA already do? Coordinate-only attacks reach over 95% attack success on point-cloud classifiers (PointBA), but the file documents two failures: such perturbations are caught by Statistical Outlier Removal and activation filters unless constrained to spectral domains (SPBA), and they have no camera-space story at all — MOBA showed that what a laser sees and what a camera sees physically diverge, so a trigger optimized in one modality silently fails in the other. Our experiment on fusion pipelines, where SPBA-style triggers serve as an ablation, will quantify exactly this collapse.

Why not just optimize geometry and appearance sequentially? This is the honest threat to the proposal, because sequential design is far cheaper and might suffice if the geometric footprint is tiny. The evidence says otherwise within benign reconstruction: decoupled pipelines suffer texture stretching and misalignment, and joint optimization cut Chamfer-$L_1$ error by 47.3% over decoupled baselines (OMeGa); Text2Mesh and DreamMesh show appearance cannot push geometry into physically plausible relief when the two are decoupled, leaving a trigger that reads as an artificial overlay. We settle it empirically rather than rhetorically: the decoupled ablation above is run at matched compute, and if it matches joint performance on every metric, the joint machinery is not earning its cost and we will report that.

Why not just use a viewpoint trigger like GaussTrap? GaussTrap implants a backdoor in 3D Gaussian Splatting that fires only at one camera pose, reaching 40.85 PSNR at the attack viewpoint while holding roughly 32 PSNR on normal views — elegant, but the trigger is an external camera pose in a static scene, not a property of any object. A moving agent on an arbitrary trajectory will essentially never hit it, and the backdoor has no physical carrier that persists across viewpoints. Our trigger rides on the object's surface, so any sensor that observes the object — from any angle — sees it.

Proposed Experiments

The primary testbed is KITTI camera-LiDAR fusion, the benchmark shared by BadFusion and MOBA, with MVX-Net as the main victim and Focals Conv-F and EPNet for transfer. Metrics are attack success rate, clean mAP (the reference points being BadFusion's 88.65% clean mAP and 95.28% attack success on MVX-Net), trigger Chamfer Distance against the clean object mesh, and survival rate under Statistical Outlier Removal and curvature-based filters; our success criterion is attack success rate within a few points of BadFusion's while keeping Chamfer Distance near zero and clean mAP within one point of the unattacked model. The secondary testbed is ModelNet40 with PointNet, PointNet++, and DGCNN, directly comparable to SPBA's 97.58% attack success and 1.08 Chamfer Distance, plus ShapeNetPart where the file reports no numbers for either attack and we would establish the first. Stealth is measured by rendered-view PSNR/LPIPS between trigger-bearing and clean objects across novel viewpoints, and by a human study on trigger detection. The decisive checks are three: the joint-versus-sequential ablation at matched compute, which isolates whether co-optimization is load-bearing; the AND-trigger audit, where defenders inspecting geometry alone or appearance alone should find nothing while the combined audit fails; and a small physical validation with a manufactured embossed-texture object under a real camera and depth sensor, since MOBA demonstrated that digital-to-physical transfer is where plausible attacks die. If the joint trigger matches single-modality attack success while surviving the filters that defeat each of them, the conclusion is that sensor fusion pipelines inherit a new attack surface precisely from the geometry-appearance coupling their clean counterparts exploit.