Error Mining for Furniture Interpenetration

Generators of 3D rooms place furniture whose meshes pass through each other: a chair sinks into a desk, a nightstand clips into a bed. The usual fixes treat this as one aggregate number to push down, either by cleaning the whole training set or by adding collision penalties and rejection filters. This proposal instead asks which parts of a scene cause the collisions. We borrow error mining from wide-coverage grammar engineering, a statistical method that finds the specific dictionary entries responsible for parse failures in millions of sentences. We apply it to large sets of generated and training scenes and ask which object categories, category pairs, grammar rules, or size ranges are responsible for mesh-level interpenetration. The output is a ranked list of culprits, and that list supports targeted repair.

The second idea comes from the same grammar tradition. Parsing research found that theoretical worst-case cost can badly mislead about real behaviour: unification parsers are exponential in the worst case but near-quadratic on real sentences. Scene-generation papers routinely argue that rejecting colliding samples becomes hopeless in dense rooms because the success probability "drops exponentially" with object count. Nobody has measured this. We propose to measure how collision rates actually scale with the number of objects and with grammar size. We then use the mined culprits to explain the curve that results.

Core Claim

Mesh interpenetration in generated scenes is concentrated, not diffuse. A small set of category pairs and attribute ranges, some inherited from contaminated data and some introduced by the generator, accounts for most failures. A blame-sharing error-mining algorithm should recover these culprits better than simple failure-rate counts, because co-occurring objects confound naive counts. Repairing only the top culprits should recover most of the benefit of cleaning the whole dataset, at a fraction of the edits.

Grounding in Prior Work

Several papers motivate the gap. CASAGPT builds furniture out of small cuboid parts, generates them with a Llama-3 autoregressive model, and filters samples by 3D overlap. It also documents that 3D-FRONT contains native collisions that models reproduce, and it fixes this by globally nudging every colliding mesh apart (3DFRONT-NC). It says nothing about which collisions matter or where generator-induced errors come from. SG-VAE mines a context-free scene grammar from co-occurrence statistics and decodes it with grammar masking, so every output is a structurally valid room. Its authors concede that grammatical validity is "necessary but not sufficient": the continuous box parameters still collide. Their discovered rules give us a natural unit to which blame can be assigned.

The stochastic And-Or grammar of Human-Centric Indoor Scene Synthesis avoids pairwise checks by reasoning about human affordances and path planning. That design rests on the untested premise that object-to-object checking explodes. Our scaling study tests that premise directly rather than assuming it.

Two papers supply the method. Error Mining in Parsing Results iteratively redistributes the blame for each failed sentence among the words it contains until the estimates stabilise. It ranks suspects by suspicion times log-frequency, and it isolated 18,334 relevant suspicious forms from over 327,000 candidates on a 14.5-million-word corpus. Relating Complexity to Practical Performance supplies the empirical-scaling methodology. It measured parse time against sentence length on the 782-rule ANLT grammar and found roughly quadratic growth up to about 20 words, against an exponential worst case.

Two papers constrain the design. ContactOpt showed that forcing zero penetration hurts realism for compliant materials, and its soft contact model permits up to 2 mm of overlap. Our collision oracle therefore needs category-dependent tolerances: a pillow pressing into a mattress is not an error. The trade-off between contact and penetration catalogued for collision losses (for example, the "tolerant collision loss" of Layout-your-3D) means any repair can induce floating objects. Every evaluation therefore reports floating alongside penetration.

Research Program

The first component is a mesh-level collision oracle. For each pair of placed furniture meshes it runs exact triangle-triangle intersection tests and computes penetration depth. It also runs a support test that flags objects hovering more than a small gap above any supporting surface. Tolerances are set per material class: rigid pairs use a near-zero threshold, and compliant pairs such as bedding and cushions use a ContactOpt-style allowance. A scene "fails" if any pair exceeds its tolerance.

The second component is the miner. Each scene is described by a bag of "forms" at three levels of granularity: object categories, ordered category pairs within a distance threshold, and, for grammar-based generators, the SG-VAE production rules used together with discretised size and orientation bins of their attributes. Following Sagot and de la Clergerie, let $F_s$ be the forms in failed scene $s$ and $S_f$ the current suspicion of form $f$. Each failure distributes one unit of blame:

$$ b_{f,s} = \frac{S_f}{\sum_{g \in F_s} S_g}, \qquad S_f \leftarrow \frac{\sum_{s \ni f,\ s\ \text{failed}} b_{f,s}}{|O_f|}, $$

Here $O_f$ is the set of all scenes, failed or not, containing $f$. The update is iterated to a fixed point, and forms are ranked by $M_f = S_f \ln |O_f|$. Blame sharing matters here: a frequent, innocent object that happens to co-occur with a genuinely problematic one ends up with low suspicion.

The third component is an empirical scaling study. Suppose collisions were independent across all pairs, each with probability $p$. A room with $N$ objects would then succeed with probability $(1-p)^{N(N-1)/2}$, which falls faster than exponentially. Suppose instead that grammatical locality confines interactions to a bounded neighbourhood of size $k$. Success then decays like $(1-p)^{kN/2}$, an exponential in $N$ with a much smaller rate. We fit both forms, plus a "concentrated culprit" model in which failure is driven by the presence of specific forms, to measured success rates as functions of $N$ and of grammar rule count. Comparing these fits answers the Carroll-style question about this domain.

Baselines

The naive reference point is a random ranking of forms, together with the no-repair condition in which generators are trained on raw 3D-FRONT. These establish the floor for both culprit recovery and downstream non-intersection rate.

The strongest competitor for attribution is direct failure-rate ranking: sort forms by the fraction of scenes containing them that fail, optionally weighted by log-frequency. It is the obvious thing a practitioner would compute, and it uses exactly the same oracle outputs and scenes as our miner. Removing only the blame-sharing iteration gives the key ablation. That makes this baseline also the test of whether the method's core ingredient matters. We additionally ablate the form vocabulary (categories only, versus pairs, versus grammar rules with attribute bins) to see which granularity carries the signal.

The strongest competitor for repair is CASAGPT's global cleaning (3DFRONT-NC), which edits every colliding scene. CASAGPT reports a bedroom non-intersection rate of 90.6%, against "the 70-80% range of previous methods" such as ATISS and DiffuScene. We will retrain ATISS, DiffuScene, CASAGPT and SG-VAE on three versions of the data: raw 3D-FRONT, fully cleaned 3DFRONT-NC, and our targeted repair, which edits only scenes containing the top-$k$ mined culprits. All runs use identical architectures, epochs, hyperparameter search budgets and seeds, and are evaluated with our mesh-level oracle rather than box overlap. The paper reports FID improvements but no exact FID value appears in the file, so we will recompute FID ourselves under one protocol. Beating the global-cleaning baseline means one of two outcomes. Either targeted repair matches its non-intersection rate within seed noise while editing substantially fewer scenes and without raising the floating rate, or it exceeds it on generator-induced failures, which global cleaning cannot touch.

Why not just…?

Why not just count how often each category pair collides? This is the most serious alternative, and in sparse bedrooms it may perform about as well as the miner. Its weakness is confounding. In dining and living rooms, chairs appear in nearly every scene alongside the objects that actually cause failures, so raw failure rates inflate innocent forms. The defect-injection experiment below settles this directly. If failure-rate ranking matches the fixed-point miner on recall of planted culprits across room types, we will report that error mining adds nothing beyond counting and keep only the scaling study and targeted repair.

Why not just clean the whole dataset, as CASAGPT does? Global cleaning works, and the reported jump to 90.6% shows it. It fixes data-inherited collisions, however, without saying which collisions the generator itself introduces, and it edits many scenes that may have been harmless or tolerably compliant. Our repair experiment compares the two at equal training budget. Mining generator outputs on already-clean data isolates the model-induced failures that cleaning by construction cannot address.

Why not just add a differentiable collision loss or reject colliding samples? Those are generation-time fixes, and they are complementary to this work rather than competing with it. The contact potential field of CPF and ContactOpt's soft contact model are examples. Their known failure mode is that penalties strong enough to remove overlap push objects into floating, and our oracle measures floating explicitly. Whether rejection sampling is viable depends on exactly the scaling curve we propose to measure. If measured success decays slowly with $N$, rejection is a fine baseline and the field's "exponential" argument against it is wrong. That would itself be a result.

Proposed Experiments

The first decisive check is planted-culprit recovery. Starting from 3DFRONT-NC, we inject controlled interpenetrations for chosen category pairs and size bins at varying rates, from 1% to 20% of scenes containing them. We train each generator on this data, mine its outputs, and measure recall at $k$ of the planted culprits for the fixed-point miner versus failure-rate ranking and random ranking. Every 3D-FRONT room type is included, so the effect of clutter on confounding is visible.

The second check is the scaling study. We bin generated and real scenes by object count, measure mesh-level success rates per generator, and compare the pairwise-independent, local-neighbourhood and concentrated-culprit fits by held-out likelihood. We then re-measure after removing scenes containing top mined culprits. If the curve flattens markedly, collisions are concentrated rather than combinatorial.

The third check is targeted versus global repair on raw 3D-FRONT. We report non-intersection rate, floating rate, recomputed FID, and the number of scenes edited, across all four generators.

The fourth experiment applies the miner to SG-VAE outputs at the grammar-rule level. Its purpose is to name which discovered productions have continuous attributes that collide, turning SG-VAE's "necessary but not sufficient" caveat into a specific list of rules to constrain.

On SUN RGB-D and SUNCG the oracle can only use boxes, so results there serve only as a secondary check, reported alongside the mesh-level numbers to show how much box-level checking inflates or hides collisions. CASAGPT's argument predicts this discrepancy, but no one has quantified it.