Fuzzy rule systems express predictions as readable IF-THEN statements, such as "if elevation is high and slope is steep, then the cover type is spruce." They have a well-known scaling problem. If every input is split into a few linguistic levels and every combination gets its own rule, ten inputs with three levels already need 59,049 rules. The usual fixes avoid that count in one of two ways. One is to make the model additive, so it never combines variables, which gives up the "and" that made the rules readable. The other is to arrange low-dimensional rule blocks in a hierarchy, but then someone has to decide by hand which variables belong together. That choice is the real bottleneck, and it is a judgment about which variables actually interact in the domain. That kind of judgment is what structured expert elicitation was built to produce.
We propose to run a Delphi process to decide which variables interact. A Delphi process is an anonymous, multi-round expert survey in which panelists see aggregated peer ratings and anonymized reasons between rounds. Here the panel combines a compact human group with independent LLM drafters. Its output is a sparse interaction graph whose consensus strength sets how hard each interaction is enforced in a hierarchical fuzzy model. The claim we will test is narrow and can fail. When labeled data are scarce or the test distribution shifts, an elicited interaction graph should give more accurate and more stable rule sets than an interaction graph found by data-driven search under the same rule budget. As data grow, that advantage should shrink in a measurable way. Dataset scale is therefore not a nuisance in this proposal. It is the axis along which we locate where expert structure stops paying for itself.
Core Claim
Two results in the file define the opening. KANFIS merges neuro-fuzzy inference with Kolmogorov-Arnold networks, which build a function from learnable one-variable curves. Its additive aggregation scales linearly in input dimension, and on Spambase it cut the average number of features per rule from 19.38 to 1.62. The cost is that it drops strict conjunctive semantics, and it has no channel for domain knowledge. Hierarchical Fuzzy Neural Networks With Privacy Preservation for Heterogeneous Big Data keeps conjunctions inside small blocks and scales to more than 4.2 million samples, but its feature grouping is fixed by hand. Both papers name automated hierarchy discovery as open. We build on both and change one thing: the block structure comes from an auditable consensus procedure instead of from hand-design or pure gradient pressure.
The model has additive one-variable fuzzy terms for all $D$ inputs, as in KANFIS. It also has conjunctive grid blocks, but only over the variable subsets $S$ in the elicited graph $G$. With $M$ fuzzy levels per variable, the rule count is
$$R = D\,M + \sum_{S \in G} M^{|S|},$$
which stays small as long as the panel endorses only low-order subsets. Our human panel has six people because The Human-AI Hybrid Delphi Model reports that compact six-expert panels reached thematic saturation. That paper also labels each item with one of four consensus tiers: Strong, Conditional, Operational, and No Consensus. We map these tiers onto the model in three ways. A Strong-consensus interaction becomes a block that is always present. A Conditional interaction ("it depends on X") becomes a block gated by the named condition and penalized for sparsity, so the data can switch it off. A non-consensus interaction is excluded. HAH-Delphi treats the "it depends" answers it collects as a limitation of numerical averaging. Here those answers become model structure.
Research Program
The elicitation stage is designed around what is known about multi-agent deliberation. Voting or Consensus? Decision-Making in Multi-Agent Debate found three things. Consensus protocols filter factual errors better than voting, by 2.8% on knowledge tasks. Independent drafting by every agent before aggregation added 3.3%. Longer discussion caused drift and sycophancy. Whether two variables interact is a factual question about the domain, so we use a consensus rule with all-agents drafting, cap the process at two rounds, and keep panelist identity anonymous to other panelists.
In round one, LLM drafters and the human panelists independently propose and rate candidate interactions from variable names and documentation, each with a written rationale. Coordinators then compute item-level retention with a content-validity-ratio style threshold, the same statistic Delphi instrument studies use to keep or drop items, as in VISION-Cog. In round two, panelists see the aggregate ratings and anonymized rationales and either revise or justify their outlier positions. Rationales are coded into consensus tiers. In HAH-Delphi a human facilitator does this coding; we semi-automate it and audit a sample by hand.
The modeling stage trains the hybrid additive-plus-block model end to end with the gating and sparsity penalties described above. Because we expect this structure to over-constrain the model at scale, as noted in The Scalability Gap in Specialized Machine Learning Models, we add one data-driven escape valve. With enough data, a held-out interaction screen may propose additional blocks, under the same total rule budget $R$. A key analysis output is how often that valve fires as a function of $n$.
Baselines
The naive reference points come first. Flat grid ANFIS is run where it is feasible, which means only on low-dimensional data such as CCPP; there, KANFIS reports RMSE 3.9542 for type-1 ANFIS. Alongside it we run an unconstrained gradient-boosted tree ensemble as the accuracy ceiling that interpretability is paying against. On ForestCover and the other datasets we use, we found no reported result on Lacuna for any of these methods, so all of them will be run fresh.
The strongest existing interpretable method is KANFIS with its sparse masking and distinctiveness regularization. On CCPP it reports RMSE 3.9313, and our model must not be worse there. On the higher-dimensional sets it is the method that learns structure purely from data.
The second structured competitor is the hierarchical fuzzy network with grouping chosen by hand or at random. On EEG-DD, PP-HFNN reports mAP 84.98% against 77.44% for standard fuzzy networks. That result shows hierarchy helps, but it says nothing about which hierarchy is best.
The most dangerous competitor is the same hybrid model with its graph chosen by data-driven pairwise interaction screening on the training split. A second automated competitor gives the graph to a single LLM, or takes a majority vote over sampled LLM graphs, with no iteration and no humans.
The ablations isolate the proposal's ingredients. One uses a random graph matched in size and order. Another uses the Delphi graph with tiers flattened so that every endorsed interaction is a hard block. A third uses LLM-only Delphi without the human panel.
All methods share the same splits, the same rule budget $R$ (we compare at fixed $R$ and also report Pareto curves), the same fuzzy level count $M$, and the same number of hyperparameter trials. The data-driven screen sees only training data. We count beating KANFIS and the data-screened graph as two conditions both holding. First, on rows where n is at most a few thousand, accuracy at equal rule count must be statistically higher across ten resampled training sets. Second, under the distribution-shift splits the accuracy loss must be smaller. Matching them in-distribution while using fewer features per rule counts as a partial result, not a win.
Why not just…?
Why not just let the data choose the interactions? This is the real threat. With enough samples, interaction screening should find true interactions on its own. That is why the main experiment is a learning curve and not a single number. We predict an advantage for the Delphi graph at small n and under shift, and convergence at full scale. If the data-screened graph matches the Delphi graph across the whole curve and on the shifted splits, we will report that elicitation adds nothing beyond auditability.
Why not just ask one LLM for the interaction graph? An Analysis of Concept Bottleneck Models found that LLM-generated concept annotations agreed with human experts only 21.8% of the time. That evidence concerns concept labels, though, not interaction structure, so it does not settle the question here. The single-LLM and majority-vote baselines test it directly. Comparing LLM-only Delphi with the hybrid panel shows separately what iteration contributes and what the humans contribute.
Why not just use KANFIS sparsity and skip structure entirely? KANFIS already produces short rules, but they are additive, so they cannot express "A and B together." The neuro-symbolic shortcut benchmark shows that models trained only on the task objective often reach high task accuracy with degenerate internal concepts. We measure whether this happens here by checking rule stability across seeds and by having held-out experts who did not sit on the panel rate rule plausibility.
Proposed Experiments
The primary testbed is ForestCover. Its 54 cartographic variables have physical meaning that ecologists can reason about, and it has about half a million rows, which allows learning curves from 100 rows up to the full set. It also allows a natural shift split: train on some wilderness areas and test on held-out ones. Spambase, Breast Cancer Wisconsin, and Parkinsons provide small-n medical and text checks with interpretable feature names. CCPP is a sanity check against the KANFIS RMSE. EEG-DD and the Ethylene-CO sensor set test whether the elicited structure stays cheap at scale against PP-HFNN, using EEG-DD mAP and EC training time (PP-HFNN reports 1,242 s distributed) as reference points. We leave out APS Failure because its anonymized features make elicitation meaningless; that exclusion marks a stated boundary of the method.
We report accuracy or RMSE, rule count, features per rule, the fraction of Conditional blocks that the data turn off, the n at which the data-driven valve starts adding blocks, and the human time spent on the panel.
The decisive check is the crossover. At a fixed rule budget, the Delphi graph should beat the data-screened graph by a margin that shrinks as $n$ grows. The margin should persist on the held-out wilderness areas. Flattening the tiers should cost accuracy at large $n$, because rigid priors over-constrain once data are plentiful. If all three patterns hold, consensus strength is a usable measure of how far to trust domain structure. If none holds, the field learns that interaction structure in these domains is better found from data than from experts.
References
- KANFIS: A Neuro-Symbolic Framework for Interpretable and Uncertainty-Aware Learning
- Hierarchical Fuzzy Neural Networks With Privacy Preservation for Heterogeneous Big Data
- The Human-AI Hybrid Delphi Model: A Structured Framework for Context-Rich, Expert Consensus in Complex Domains
- Voting or Consensus? Decision-Making in Multi-Agent Debate
- Feasibility, comprehensibility and acceptability of the VISION-Cog, a novel tool to assess cognitive impairment in visually impaired older adults: a cross-sectional pilot study in Singapore
- An Analysis of Concept Bottleneck Models: Measuring, Understanding, and Mitigating the Impact of Noisy Annotations
- A Neuro-Symbolic Benchmark Suite for Concept Quality and Reasoning Shortcuts