Teaching Small Decision Trees to Segment Plant Organs from a Few Clicks

Measuring plant architecture in 3D (leaf length, stem height, leaf count) starts with a step that nobody enjoys. Someone has to label every point of a scanned plant as stem or leaf. Pheno4D, the main public benchmark, needed about 260 million hand-labeled points across 14 plants. The soybean study behind PVSegNet stopped at 60 plants because labeling ran out (Pheno4D; Cui et al.). A plant scan, however, carries a strong free signal that current segmenters ignore during training: physical connectivity. Points that touch along a leaf surface almost always belong to the same organ.

We propose to exploit that signal with a small, globally optimized decision tree. A breeder places a handful of clicks per plant. Those labels spread along the 3D neighbor graph of the point cloud, and at the same time a sparse oblique decision tree (a tree whose splits each use a weighted combination of a few features) is fit to per-point geometric descriptors. The two steps alternate until the tree's predictions agree with the spread labels. The finished tree takes only local shape descriptors as input, so it runs on a CPU and labels new plants without building any graph. Each of its splits is a readable rule, for example "roughly vertical normal and low planarity means stem."

Core Claim

The method is a modification of Laplacian Tree Alternating Optimization. LapTAO trains decision trees semi-supervised by alternating two steps. In the first, a label step diffuses continuous stand-in labels over a graph. In the second, a tree step refits a tree to those stand-in labels using TAO. On MNIST it reaches about 5% error with 1–3% of labels. In LapTAO the graph is built in the same feature space the tree sees, and it is a dense affinity graph that the research file flags as prohibitive beyond a million points.

Our change is to build the graph in physical 3D space while the tree sees only descriptors computed around each point. This makes the graph a sparse k-nearest-neighbor structure with cost linear in the number of points. It also means the graph carries information the tree cannot see directly, namely which points are connected to which. The tree is then forced to turn the connectivity prior into rules about local shape, rules that transfer to unseen plants.

The claim is testable. At click-level supervision, around 0.1–1% of points, this joint procedure should produce a segmenter of a few thousand parameters. Its performance on unseen Pheno4D plants should approach fully supervised deep networks and should beat simpler ways of combining the same graph with a tree. If it does not beat those simpler combinations, then global tree optimization adds nothing here, and we will report that.

Why This Pairing, and What Prior Work Plays Which Role

The following work supplies the method:

  • TAO. Alternating optimization of decision trees refines all nodes at a given depth in parallel. Each node is fit by a sparse linear classifier on only the samples whose routing matters, and dead branches are pruned as a side effect. On MNIST it cut oblique CART error from 11.03% to about 5.6% while pruning hundreds of nodes.
  • LapTAO. It supplies the alternation between the label step and the tree step described above.

The following work motivates the gap:

  • Pheno4D and PVSegNet. They document the cost of labeling.
  • Deep segmenters. PointSegNet and PVSegNet reach 93.73% and 92.10% mean intersection-over-union (mIoU) with full labels. PointSegNet does this with 1.33M parameters, and neither has been tested with sparse labels.
  • Fractual Projection Forest. It shows that hand-built geometric features plus a greedy random forest can classify whole point clouds in under a second on a CPU. It works only on rigid CAD objects, uses greedy trees, and does not attempt per-point segmentation.

The following work constrains the design:

  • Auto3DPheno. It already extracts maize seedling traits with no labels at all, using skeleton contraction and bifurcation rules (0.76% relative error on plant height, 1.26% on leaf length). So label-free geometry is a serious competitor on simple, unbranched architectures. For that reason our main target is tomato, where petioles and branching break single-stem skeleton heuristics.
  • Statistical Optimality of Top-Down Decision Tree Learning. It argues that greedy splits can be far from optimal. This is a reason to expect a gap, not proof that one exists on point features.

Research Program

Each point receives a descriptor vector $x_n$ built from standard local geometry:

  • covariance eigenvalue ratios at three neighborhood radii (how line-like, plane-like or blob-like the neighborhood is);
  • how vertical the surface normal is;
  • height above the estimated plant base;
  • horizontal distance from the plant's vertical centroid axis.

The graph Laplacian $L$ is built from a k-nearest-neighbor graph over 3D coordinates. Edge weights decay with distance and are reduced where neighboring normals disagree.

Following the LapTAO construction, we solve:

$$ \min_{T, Z}\; \sum_{n \in \mathcal{S}} \ell(y_n, z_n) \;+\; \alpha\, \mathrm{tr}(Z^\top L Z) \;+\; \mu \sum_{n} \lVert z_n - T(x_n) \rVert^2 $$

The symbols are:

  • $\mathcal{S}$ is the small set of clicked points and $y_n$ are their organ labels;
  • $Z$ holds one continuous stand-in score per class for every point;
  • $T$ is a multiclass sparse oblique tree;
  • $\alpha$ controls how strongly labels spread along the 3D graph;
  • $\mu$ ties the stand-in labels to the tree's output, and it is increased over iterations.

With $T$ fixed, the $Z$-step is a sparse linear system solved by conjugate gradients, at a cost that scales with the number of graph edges. With $Z$ fixed, the $T$-step is one TAO pass, warm-started from the previous tree, with an $\ell_1$ penalty so that each split uses only two or three descriptors.

Training runs on voxel-downsampled clouds of roughly 50k points each. The tree is then applied to every raw point.

Leaf instances are obtained as connected components of leaf-labeled points on the same graph. Leaf length is measured with the same downstream geometry used by the Pheno4D baseline, so that differences in trait error come from segmentation alone.

We will simulate supervision as a breeder would actually provide it: one to three clicks per organ instance plus a few on the stem. We will also report uniformly random labeled fractions of 0.1%, 1% and 10% for comparison with the LapTAO literature.

Baselines

The naive reference point is a tree trained only on the clicked points, with no graph. We will run both CART and plain TAO on the same descriptors. This measures what the connectivity prior contributes at all.

The strongest simple competitor, and the one we most need to beat, is a two-stage pipeline:

  • first, spread the clicks over the same 3D graph with the same $\alpha$, solving only the label step;
  • then, train a greedy random forest and an XGBoost model on the spread labels, using the same descriptors.

This pipeline uses exactly our graph and features. It removes only the joint alternation and the oblique, globally refined tree. We will give it the same hyperparameter search budget as our method, measured in grid points over $\alpha$, depth, and number of trees. We will also match it to our CPU time. Beating it means higher mIoU on held-out plants across leave-one-plant-out folds, using paired tests over folds, at comparable or much smaller model size.

A second competitor applies graph smoothing at test time to the per-point outputs of a greedy forest trained on clicks. This gets connectivity at test time rather than training time.

The deep reference is PointSegNet trained on the same clicks with the same label spreading used as pseudo-labels. Its fully supervised published numbers serve as a ceiling. On Pheno4D, the fully supervised LatticeNet baseline reports:

  • 98.8% semantic mIoU on maize and 95.1% on tomato;
  • symmetric best Dice of 80.6 for maize instances;
  • a mean tomato leaf-length error of 8.9 mm (Pheno4D).

On the soybean data, PVSegNet reports 92.10% mIoU and 83.47% AP@50 with full labels (Cui et al.).

The Pheno4D split behind the LatticeNet numbers may differ from ours. We will therefore retrain at least one deep model under our leave-one-plant-out protocol rather than compare against the published figure alone.

Three ablations isolate the key ingredient:

  • axis-aligned versus sparse oblique splits;
  • a graph built in descriptor space, as in original LapTAO, versus our 3D graph;
  • joint alternation versus the two-stage pipeline above.

Why not just…?

Why not just train a random forest on geometric features, as Fractual Projection Forest does? Fractual Projection Forest shows such forests are fast and competitive for whole-object labels. With only clicks, though, a forest sees a few hundred labeled points per plant and nothing about connectivity. The naive baseline will quantify how much that costs. Separately, TAO regression forests matched or beat a 100-tree random forest (about 230k parameters) with 30 trees (about 8k parameters) on Abalone. That is tabular evidence, not point-cloud evidence, that globally refined trees can be much smaller at equal accuracy.

Why not just spread the labels over the graph and then train any classifier? This is the real threat. It may capture most of the benefit, because the graph could be doing the work. The two-stage baseline is designed to settle this question. Our argument for joint training is that the tree step feeds back into the label step. In regions where the geometry is unambiguous, the tree's predictions correct spread labels that leaked across organ junctions such as a petiole-stem contact. A one-shot propagation cannot make that correction. If the two-stage forest matches us, the publishable conclusion becomes "the 3D graph is what matters." We will frame the result that way rather than argue around it.

Why not just fine-tune a deep point network on sparse labels? Deep networks win with full labels, and we do not dispute that. What is missing is any reported result for plant point segmentation at click-level supervision. So the deep baseline at matched supervision is a genuine experiment, not a straw man. If PointSegNet with pseudo-labels matches our accuracy, our remaining advantages are CPU inference and readable splits. Whether those matter for field robots is a deployment question, not an accuracy win.

Why not just use label-free skeleton geometry? On maize seedlings, Auto3DPheno already achieves low trait errors without any labels. We therefore do not claim to improve there. We include it as a baseline on the maize data and focus our claims on tomato, where branching makes its bifurcation rules less applicable.

Proposed Experiments

Main experiment. We evaluate on Pheno4D tomato and maize with leave-one-plant-out splits, holding out each plant's entire time series. This tests generalization to unseen plants, not unseen days of a seen plant. We report mIoU and symmetric best Dice at each supervision level, together with the resulting leaf-length error in millimeters.

Decisive checks. There are three:

  1. whether our method beats the two-stage graph-plus-forest pipeline on held-out plants at click-level supervision;
  2. whether its tomato leaf-length error falls within a few millimeters of the fully supervised 8.9 mm while using at most 1% of the labels;
  3. whether the sparse oblique splits beat axis-aligned ones at equal depth, which is the test of whether organ boundaries really are slanted in descriptor space.

Transfer test. We repeat the protocol on the 60-plant soybean set (stem versus pod, with pod counts from connected components). This checks that the result is not specific to Pheno4D.

Cost and interpretability. We will record CPU inference time and parameter count. We will also have the splits read out by a plant scientist to see whether the learned rules match botanical intuition.