Who Draws the Tree? Community-Elicited Object Hierarchies for Hyperbolic Recognition Across Regions

Image classifiers recognize a stainless-steel range in a European kitchen far better than a clay pot on a wood fire in a rural home, even though both are ways of cooking. Most fixes for this geographic gap either buy more photographs from under-represented places, which is expensive, or rebalance losses after training, which leaves the category system untouched. This proposal targets a cheaper and less studied lever: the structure of the label space. We ask residents of under-represented regions to decide which everyday objects count as "the same kind of thing," and how those kinds nest into broader functions such as cooking, storing water, or lighting. We then build a classifier whose internal geometry is organized around that community-drawn tree.

The geometry is hyperbolic embedding, meaning representations placed in a curved space whose volume grows exponentially outward, much as the number of nodes in a tree grows with depth. That property lets a tree sit in a few dimensions with little distortion (Nickel and Kiela). It also has a cost: hyperbolic models are known to degrade sharply when the tree they are given is misspecified (LLM-Guided Restructuring). Our central bet follows from that sensitivity. Because hyperbolic models amplify whatever hierarchy they are handed, they are the setting where a better, community-sourced hierarchy should pay off most. They are also where the Western-default hierarchies inherited from WordNet-style taxonomies should hurt most.

Core Claim

For recognizing household objects across regions and incomes, the source of the label hierarchy interacts with the geometry of the embedding. Specifically, we claim that a hierarchy elicited through structured participation with residents of the regions where models fail, embedded in hyperbolic space, will do three things better than either the same hierarchy in flat Euclidean space or a standard or LLM-generated hierarchy in hyperbolic space. First, it will shrink worst-region error. Second, it will make the model's remaining mistakes less severe, as judged by residents who were not part of the elicitation. Third, it will help most in the low-data regime typical of under-represented regions. We further claim that eliciting the tree costs a small fraction of commissioning an equivalent amount of new imagery. We test this directly against a budget-matched image-collection arm.

What is new is the specific pairing. Community participation here shapes a model's inductive bias, the built-in assumptions that guide how it generalizes, rather than supplying photos or labels. The payoff is predicted to depend on geometry, which makes the claim falsifiable through an interaction test rather than a single headline number.

Where This Sits

Three lines of work motivate the gap. HierLoc embeds a Country→Region→Subregion→City tree of place prototypes in the Lorentz model of hyperbolic space and reports a mean geodesic error of 861 km on OSV5M. That is a 19.5% improvement over the strongest prior baseline, alongside large efficiency gains. Its tree is taken wholesale from administrative gazetteers, and its authors report that accuracy stays markedly lower in sparsely photographed regions. We reuse HierLoc's machinery: prototype nodes, a hierarchy-aware contrastive loss, and coarse-to-fine search. We change the task from geolocation to object recognition and change who defines the tree. We deliberately avoid geolocation, where better accuracy on community photos raises surveillance concerns that a participatory project should not ignore.

GeoDE and Dollar Street document the gap and supply our benchmarks. GeoDE commissioned photos of 40 object classes from paid contributors in six regions. It finds models about 7% worse on African than European images, with classes like "dustbin" at 37.3% accuracy, at a cost of $0.87 per image. Dollar Street photographed 404 homes in 63 countries and tagged each with household income. Both treat categories as a flat list and treat participants as photo collectors. GeoDE required a smartphone, which excluded the poorest households. We keep their images and change the participants' role.

On the participatory side, Community Based System Dynamics shows how top-down problem formulation falls into a "proxy trap," but leaves open how community input becomes a model component. Accountability Driven Development turns values into testable natural-language scenarios, and scaffolding methods such as comicboarding (Kuo et al.) show that non-experts can give fine-grained technical feedback. We borrow the scenario format and the scaffolding. We then close the "translation bottleneck" that Holstein et al. and others identify, by making the output of participation a concrete object: a tree plus a test suite. That object maps mechanically onto hyperbolic distances and evaluation checks.

Two papers constrain the design. LLM-Guided Restructuring suggests that a language model can repair hierarchies cheaply. That makes it the most dangerous competitor, and it runs as a full arm rather than a strawman. The gender-bias work on hyperbolic word embeddings shows that bias survives in hyperbolic geometry. Curvature alone is therefore not a fix, which is why the claim concerns the tree, not the space.

Research Program

The participatory component runs facilitated card-sorting sessions through local partner organizations in the three GeoDE regions with the largest reported deficits, Africa, West Asia, and Southeast Asia. A European comparison group runs in parallel so that "any human sort" can be separated from "sorts by residents of the failing regions." Participants receive printed photo cards drawn from GeoDE and Dollar Street, so no smartphone is needed. They group cards, name the groups, and nest groups under broader functions. Facilitators then help each group write a handful of acceptance scenarios in the Given/When/Then form. An example: "Given a photo of a jerrycan used for household water, when the model is wrong, then its guess should still be a water-storage object and not a fuel container." The individual sorts are merged into one consensus tree by standard hierarchical agreement clustering. Disagreements are kept as soft multi-parent links rather than resolved by fiat.

The modeling component freezes a CLIP-style image encoder for all arms. A small projection head then maps features either into the Lorentz model or into Euclidean space. Class and internal-node prototypes are trained with a hierarchy-aware contrastive loss adapted from HierLoc's loss, with tree distance replacing its haversine term. The hyperbolic arm places general functions near the origin and specific variants toward the boundary. At inference it descends the tree coarse-to-fine. Freezing the encoder keeps compute small and ensures that every difference comes from label structure and geometry, not from a larger backbone.

The hierarchy sources form one axis of the design. We compare no hierarchy (flat), an ImageNet/WordNet-derived tree, an LLM-restructured tree following the LLM-guided method, a data-driven tree from agglomerative clustering of visual features, the European participants' tree, and the Global South participants' tree. Crossing these six sources with the two geometries gives the core grid. We run it at full training data and at a few-shot setting of 5 and 20 images per class for the held-out region, which mimics data scarcity.

Baselines

The naive reference point is zero-shot CLIP and the pretrained ImageNet models evaluated as-is on GeoDE and Dollar Street. The reported numbers set its floor: 18.0% top-5 accuracy on the lowest Dollar Street income quartile versus 57.4% on the highest (Dollar Street), and a 7% Africa–Europe gap on GeoDE (GeoDE). Any trained arm should clear this easily, so it anchors scale rather than serving as the real test.

The strongest existing competitor is flat fine-tuning on geographically diverse data. Dollar Street reports that fine-tuning a ResNet brings accuracy to roughly 75% averaged across income quartiles. GeoDE reports that adding its images to ImageNet training improves Dollar Street accuracy by 9% and GeoDE test accuracy by 21%. We rerun flat fine-tuning of the same projection head on the same frozen encoder, with the same training images, the same splits, and the same hyperparameter search budget of 30 trials per arm. On in-distribution top-1 accuracy we expect our method only to match it. Beating it means a statistically significant gain across five seeds on at least two of three outcomes: worst-region accuracy, few-shot accuracy on the held-out region, and severity of mistakes as judged by held-out residents. Parity on everything counts as a failure of the proposal. I found no reported result on Lacuna for any hyperbolic or hierarchy-aware classifier on GeoDE or Dollar Street, so those arms have no external number to beat and are compared only internally.

The key ablation isolates the participatory tree in two ways. Holding geometry fixed at hyperbolic and swapping in the WordNet, LLM-restructured, visual-cluster, and European-participant trees tests whether it matters who drew the tree. Holding the Global South tree fixed and switching to Euclidean space tests whether geometry amplifies the tree's benefit. A budget-matched data arm then spends the full elicitation cost on additional GeoDE-style images at $0.87 each and trains the flat model on them. Across all these comparisons every arm shares encoder, training images, compute, and tuning budget.

Why not just…?

Why not just fine-tune a flat classifier on Dollar Street and GeoDE? This is the real threat. The reported roughly 75% cross-quartile accuracy after fine-tuning shows that flat training closes much of the income gap once diverse data exist. We do not expect to beat it on in-distribution accuracy. The argument for a tree is about what flat training cannot give. One is transfer to a region with only a handful of images, where shared parent nodes should carry information across regions. The other is mistakes that residents consider sensible. If the few-shot and severity experiments show no gain over flat fine-tuning, the hierarchy is not earning its keep and we will report that.

Why not just ask a language model to build or repair the hierarchy? It is nearly free, and LLM-Guided Restructuring shows it reduces hyperbolic distortion. Language models, however, inherit the same web-text geography that GeoDE and DIG In show skews toward Western, showroom-style concepts. We have no direct evidence yet on whether that skew reaches object taxonomies, so the LLM tree runs as a full arm. If it matches the Global South participants' tree on worst-region accuracy and resident-judged severity, the honest conclusion is that participation is not needed for this purpose. That would still be useful, as a cheap default validated by communities.

Why not use the same community tree in ordinary Euclidean space? Hierarchical softmax and tree-aware losses work in flat space, and a large Euclidean dimension has ample raw capacity. The theory predicts crowding of fine-grained leaves (Nickel and Kiela), and HierLoc reports its hyperbolic hierarchy gives a 43% subregion-level gain over flat retrieval. With only 40 to a few hundred classes, however, crowding may simply not bite. The Euclidean arm with the identical tree settles this. If it matches the hyperbolic arm, the contribution reduces to the participatory tree, and we would present it that way.

Why not spend the money on more photos? GeoDE's per-image cost makes this a concrete comparison rather than a slogan. The budget-matched data arm answers it directly. If more images beat the tree at equal cost, image collection wins on this benchmark.

Why not prompt CLIP with the region name? For generation, DIG In documents that adding place names makes models drop the requested object. Classification may behave differently, so zero-shot CLIP with region-augmented class prompts is included alongside the naive baseline.

Proposed Experiments

Evaluation uses GeoDE's six regions and Dollar Street's income quartiles. Dollar Street topics are mapped to the nearest GeoDE-compatible nodes for trees that cover them. We report four quantities. The first is worst-region accuracy and its gap to the best region. The second is lowest-quartile top-5 accuracy. The third is scenario pass rate on the participants' Given/When/Then suite. The fourth is severity of mistakes. Following WorldStrat's argument that averages mislead in global evaluation, we also report the spread of per-region performance rather than only its mean.

Severity is where circularity lurks, since scoring errors by distance in the participants' own tree would favor that tree. We therefore measure severity in two ways that do not use it. The first is distance in every other tree, averaged. The second is pairwise judgments collected in fresh sessions with residents who did not help build the tree. They see a true object and two wrong predictions and pick which mistake is worse. Scenario suites are split so that the scenarios used for evaluation were written by different groups from those whose sorts formed the training tree.

The decisive check is an interaction effect in the grid: the Global South tree's advantage over the WordNet and LLM trees must be larger under hyperbolic geometry than under Euclidean. That is the prediction that separates "hyperbolic geometry amplifies community structure" from either ingredient acting alone. A second decisive check is the leave-one-region-out few-shot test. Trees elicited in two regions should improve few-shot accuracy on the third, which shows the structure carries information beyond the elicitors' own photos. A third is cost. Elicitation will be run for an estimated few thousand dollars in facilitation and compensation, and its gains must exceed those of the budget-matched image arm. If the interaction is absent and the LLM tree ties the participatory one, we will say so plainly: a community-drawn tree is a legitimacy benefit, not an accuracy one, on these benchmarks.