Many prediction problems involve entities with numbers attached, connected by relations that also carry numbers. A protein binding site is a set of atoms with 3D distances between them. A river sample is a set of observed organisms with abundances. Logic-rule learners that work directly on such relational data, a family called inductive logic programming (ILP), can state rules like "the site binds hexose if there exists an oxygen atom within 3.1 Å of a charged residue." They can only do this, however, if someone first decides where "within 3.1 Å" draws its line. Today that line is either fixed by hand, set by a static binning heuristic, or found by greedy search one test at a time. Flat, single-table rule learners have recently shown that learning such cut-points jointly with the rules, by gradient descent, beats static binning. Relational rule learners have not adopted this idea. They either ignore continuous attributes entirely or pre-bin them.
This note proposes to learn cut-points inside first-order rules, meaning rules whose variables range over entities and are bound by "there exists." It also argues that the cost of discretization is structurally worse in this setting than in flat tables. When a rule says "there exists a neighbour Y with distance below t," every one of the many candidate neighbours near the threshold can flip the whole rule. The resulting error should grow with how many bindings a quantifier ranges over. We propose to measure this effect, which we call the relational discretization tax. We then propose a threshold mechanism designed around it and test whether it beats the cheap alternatives, fine static binning and greedy relational trees, where the theory says it should.
Core Claim
Discretization loss in relational rules is governed by quantifier fan-out, not just by bin resolution. Take a rule body with an existential variable over $n$ candidate bindings. Suppose each binding independently falls in an ambiguous band around the threshold with probability $p$. Then the chance that the existential's truth value is threshold-sensitive is
$$ P_{\text{flip}} = 1-(1-p)^{n}, $$
which grows toward one as $n$ grows. This follows directly from the rule being an OR over bindings. Static bins fix $p$ by resolution alone and ignore $n$. A greedy learner picks one threshold at a time while holding the bindings fixed. Our claim is that a threshold learner which is aware of the quantifier should shrink this sensitivity, and that its advantage over static binning should grow with $n$. Such a learner sets cut-points against the distribution of bindings that actually appear in rule groundings, rather than against the marginal distribution of the feature, and it receives gradient through the existential. The claim fails if fine static binning plus sparsity matches the learned thresholds at high fan-out.
Research Program
The host architecture is the Relational Deep Concept Reasoner from Relational Concept Bottleneck Models. That model runs message passing over ground facts, passes them through a bottleneck of interpretable relational concepts, and outputs explicit Horn rules (if-then rules over variables). It implements "there exists" as max-pooling. It matches black-box graph networks on citation graphs and knowledge graphs and supports test-time corrections of concepts. It was evaluated only on predicates that are categorical by definition. We keep its rule output and its support for interventions. We add numeric atoms: unary ones such as $\text{charge}(X) > t$ and relational ones such as $\text{dist}(X,Y) < t$, each with its own learnable ordered thresholds.
The threshold module takes three ideas from Learning to Binarize Continuous Features for Neuro-Rule Networks (AutoInt). AutoInt learns interval boundaries as positive increments so they stay ordered, uses a soft, temperature-controlled selection during training with a hard step and a straight-through gradient, and adds a soft $K$-means loss that keeps boundaries anchored to the data. We change it in three ways, and each change is testable.
The first change concerns what the anchoring loss is computed over. It uses the distribution of numeric values over groundings, weighted by each grounding's current soft contribution to the rule, rather than over entities. The population a relational threshold must separate is tuples such as (site atom, neighbour atom), and that population differs sharply from the marginal distribution of distances.
The second change replaces the hard max during training with a tempered log-sum-exp over bindings. Gradient then reaches every near-threshold binding rather than only the current maximizer. We anneal to the hard max so that the final rules are exact.
The third change removes AutoInt's requirement to fix the number of intervals $K$ in advance. We start from an overcomplete set of candidate cut-points per numeric atom with gates in the style of Gumbel-Softmax. We borrow this gating from how Neuro-Symbolic Hierarchical Rule Induction selects invented predicates. A sparsity penalty then chooses how many cut-points survive.
The training objective is
$$ \mathcal{L} = \mathcal{L}_{\text{task}} + \lambda_c \,\mathcal{L}_{\text{anchor}}^{\text{grounding}} + \lambda_s \sum_{a} \lVert g_a \rVert_0, $$
where $\mathcal{L}_{\text{task}}$ is the prediction loss of the hard-thresholded rules, $\mathcal{L}_{\text{anchor}}^{\text{grounding}}$ is the soft clustering loss computed over grounding-weighted values, $g_a$ is the gate vector over candidate cut-points for numeric atom $a$, and $\lambda_c, \lambda_s$ are weights.
The method also offers one capability that no current system provides: threshold interventions. An expert can overwrite a learned cut-point, for example set 3.1 Å to a known hydrogen-bond distance. The corrected value then propagates through the rules exactly, which extends the concept-intervention argument of Concept Bottleneck Models to numbers.
The other bridging work shapes the design in specific ways. Differentiable Rule Induction from Raw Sequence Inputs (NeurRL) learns a symbol alphabet by differentiable clustering of time-series windows. It shows that task gradients can reshape discretization, but only for propositional inputs, and its sensitivity to window length warns us that fixed-structure choices matter. GLIDR and HRI show that differentiable relational rule learners tolerate 40–50% label noise, which motivates the noise experiments. Both handle only discrete links, and we do not claim to extend their rule shapes. Falling Rule Lists explicitly leaves continuous-variable handling to future work. It shows that sparse rule models can reach an AUROC of 0.80 against 0.79 for random forests, which is the kind of parity we aim for in the relational setting.
Baselines
The naive reference point is the same R-DCR host with continuous attributes pre-binned by equal-frequency quantiles at a moderate resolution (K=5). This is what current relational concept models would require in practice. Beating it only shows that binning matters, which Information assessment on predicting protein-protein interactions already documents for flat Bayesian networks: error rose from 0.28% with continuous random forests to 6.34% with discretized Bayesian networks.
The strongest static competitor is the same host with fine quantile or entropy-based bins (K up to 32 per atom) plus the same sparsity penalty over bin-indicator atoms. This can emulate any threshold on a grid and is the cheapest way to close the gap. It receives the same tuning budget as our method, a fixed number of configurations per dataset selected on validation data. To count as beating it, our method must be better on held-out data at matched rule size (number of literals), not merely more accurate with larger rules.
The strongest existing relational method is TILDE Simultaneous Prediction of Mulriple Chemical Parameters of River Water Quality with TILDE, which builds first-order decision trees with greedy numeric threshold search. On the river data it reports an average correlation of r = 0.467 across 16 targets with a single multi-target tree. For hexose binding the reference is Aleph at 32.5% test error, with flattened baselines between 25% and 35%, from An Inductive Logic Programming Approach to Validate Hexose Binding Biochemical Knowledge. We rerun both on our splits rather than rely only on the reported figures. Beating them means lower error, or higher correlation, at equal or smaller model size.
The upper reference is a black-box graph network on the raw continuous attributes, together with a continuous random forest on propositionalized aggregate features. On the protein-interaction data the relevant published number is the random forest's 0.28% error. On the flat sanity task, Bank-marketing, the target is AutoInt's reported F1 of 77.59% (XGBoost reported 74.71%). Our module should reproduce roughly that figure when relations are absent.
Three ablations isolate the key ingredients. The first swaps grounding-weighted anchoring for AutoInt's marginal anchoring. The second keeps the hard max during training instead of log-sum-exp. The third fixes K instead of gating it. No reported R-CBM result on continuous-attribute relational data was found on Lacuna.
Why not just…?
Why not just use fine static bins with sparsity? This is the real threat, and the program is built to settle it. If our fan-out argument is wrong, fine bins should match learned thresholds everywhere. If it is right, they should match at low fan-out and fall behind as the number of bindings per quantifier grows. The synthetic sweep described below measures exactly this. If the curves do not separate, we will report the discretization-tax measurement as the contribution and drop the claim that learned thresholds are needed.
Why not just use TILDE or Aleph, which already search thresholds on the task objective? Greedy search is task-driven, so the static-binning evidence does not apply to it. Its weaknesses lie elsewhere. It picks one test at a time while holding the bindings fixed, so it cannot adjust a distance cut-point together with the attribute cut-point it is conjoined with. Symbolic search also degrades under label noise, where HRI and GLIDR report tolerance of 40–50%. We test both weaknesses directly: synthetic rules with two interacting numeric literals, and label-noise sweeps on the hexose-binding data. If TILDE matches us on both, the differentiable machinery is not justified.
Why not just flatten the data with aggregate features and run AutoInt or XGBoost? Aggregates such as "count of neighbours within d" do capture some existential rules, and on small, fixed-arity data they may win. Flattening, however, requires the aggregate thresholds d to be chosen in advance, which reintroduces the same problem one level up. The entity-mention and river studies also document information loss when variable-sized structures are squeezed into fixed vectors. We run this baseline with aggregates on a grid of thresholds, and we report where it wins.
Why not just train a graph network and extract rules post hoc? Surrogate rules approximate the model's behaviour rather than its computation, so editing a threshold in the surrogate changes nothing in the predictor. We include a distilled-surrogate baseline and measure its fidelity. We also measure the effect of the same threshold interventions applied to it. The expectation is that fidelity will be imperfect and that interventions will have no guaranteed effect.
Proposed Experiments
The decisive experiment is a synthetic relational benchmark with planted rules. Entities carry continuous attributes, pairs carry distances, and labels come from rules containing one or two numeric literals under an existential quantifier. The number of candidate bindings per example is swept from 1 to about 50, and Gaussian noise is added to the numeric values. We measure three things: held-out accuracy, the gap between the soft-trained model and its final hard-thresholded version (the relational discretization tax), and the absolute error between recovered and planted cut-points. The prediction is that the advantage of learned thresholds over fine static bins is near zero at fan-out 1 and grows with fan-out roughly as $P_{\text{flip}}$ predicts. The prediction also holds that grounding-weighted anchoring recovers planted thresholds better than marginal anchoring.
The real-data experiments use three datasets. The PDB hexose-binding sites (160 sites, with continuous 3D distances) have Aleph at 32.5% error as the reference. The river water-quality data has TILDE at r = 0.467 as the reference. The protein-interaction pairs are treated relationally, with the continuous random forest at 0.28% error as the ceiling. On each we report accuracy against rule size and the number of cut-points selected per atom. We also run label-noise sweeps at 10–40%.
Bank-marketing serves as a flat sanity check. The module should land near AutoInt's 77.59% F1 there, confirming that the relational extensions do not hurt the propositional case.
Finally, a threshold-intervention study perturbs learned cut-points on the hexose data toward, or away from, biochemically documented distances. It measures how predictions change and compares this to the post-hoc surrogate.
The note succeeds if the fan-out curve separates from fine static binning and the real-data results beat TILDE and Aleph at matched size. It fails informatively if fine static binning matches everywhere. In that case the measured relational discretization tax still tells practitioners how finely they must bin before running a relational rule learner.
References
- Relational Concept Bottleneck Models
- Learning to Binarize Continuous Features for Neuro-Rule Networks
- Neuro-Symbolic Hierarchical Rule Induction
- Concept Bottleneck Models
- Differentiable Rule Induction from Raw Sequence Inputs
- GLIDR: Graph-Like Inductive Logic Programming with Differentiable Reasoning
- Falling Rule Lists
- Information assessment on predicting protein-protein interactions
- Simultaneous Prediction of Mulriple Chemical Parameters of River Water Quality with TILDE
- An Inductive Logic Programming Approach to Validate Hexose Binding Biochemical Knowledge