Repair in the Loop: Training Ontology Matchers Against Their Own Logical Contradictions

Ontology alignment means finding which concepts in two independently built knowledge bases mean the same thing, for example "Lanugo" in one anatomy ontology and "Primary hair" in another. Neural matchers such as BERTMap score each candidate pair on its own, using a fine-tuned language model. Each ontology, however, carries logical rules, such as "every X is a Y" (subsumption) and "nothing is both a tissue and a developmental stage" (disjointness). A mapping that looks right on the surface can force a reasoner to conclude that some concept can have no instances at all. Such a concept is called unsatisfiable, and an alignment that produces one is called incoherent. Current systems handle this in two separate stages. The matcher predicts, and then a symbolic repair step deletes conflicting mappings. The repair's verdicts never reach the matcher. The repair itself is usually greedy because exact repair does not finish on large ontologies.

We propose to close the loop using a tool from training neural networks with binary weights. The Alternating Direction Method of Multipliers (ADMM) is a way to split a hard problem into an easy continuous part and an easy discrete part, joined by a running "disagreement" variable. Extremely Low Bit Neural Network used it to alternate between continuous weight updates and projection onto discrete codebooks. In our setting the continuous part is the matcher's probabilities. The discrete part is a keep/drop vector over candidate mappings, restricted to the set of selections that keep the merged ontology coherent. The projection onto that set turns out to be exactly the standard weighted repair problem, which a fast existing greedy repair can solve. The disagreement variable accumulates across rounds. It changes the weights the repair sees, so the repair can revisit early greedy mistakes, and it trains the matcher toward scores that are already logically consistent. The bet has two parts. First, iterating cheap greedy repair with accumulated disagreement should recover much of the accuracy that Completeness and optimality in ontology alignment debugging showed optimal repair has over greedy repair. Second, it should do so at roughly greedy cost, so that it also works on SNOMED CT, where the optimal debuggers in that paper did not finish within five hours.

Core Claim

Let $c_1,\dots,c_m$ be candidate mappings and $p_i(\theta)\in(0,1)$ the matcher's probability that $c_i$ is correct, where $\theta$ are the matcher's parameters. Let $z\in\{0,1\}^m$ be a keep/drop vector and $\mathcal{C}$ the set of keep vectors whose merged ontology has no unsatisfiable classes. Let $u\in\mathbb{R}^m$ be the disagreement (dual) variable. We optimize

$$ \min_{\theta,\; z\in\mathcal{C}} \; \mathcal{L}_{\text{match}}(\theta) + \frac{\rho}{2}\sum_{i=1}^m \big(p_i(\theta) - z_i + u_i\big)^2 , $$

where $\mathcal{L}_{\text{match}}$ is BERTMap's original synonym-classification loss on label pairs drawn from the ontologies, and $\rho>0$ sets how strongly the matcher is pulled toward the coherent selection. The method alternates three steps. It takes a few gradient steps on $\theta$. It then runs the z-step. Finally it updates $u \leftarrow u + p(\theta) - z$.

The z-step has a simple form. For binary $z_i$, $(a_i - z_i)^2 = a_i^2 + z_i(1-2a_i)$, where $a_i = p_i + u_i$. Minimizing over $\mathcal{C}$ is therefore the same as finding the maximum-weight coherent subset of mappings, with weight $a_i - \tfrac12$ on mapping $i$. That is precisely the repair problem studied by the optimality paper. We can hand it to LogMap's propositional Horn repair (LogMap family participation in the OAEI 2018), which translates the axioms into simple if-then clauses and checks them in linear time. The only change is that it receives adjusted weights rather than raw confidences.

The variable $u_i$ plays the role that Training Quantized Nets: A Deeper Understanding attributes to continuous shadow weights in binary networks. It accumulates small, persistent signals until they flip a discrete decision. In that paper this deterministic accumulation reached 8.21% CIFAR-10 error, while stochastic rounding reached 20.56%. Two cases matter here. A mapping that greedy repair keeps deleting despite high matcher confidence builds negative pressure, and the matcher learns to lower its score. A mapping the repair keeps despite a mediocre score builds positive pressure, and later rounds may re-admit related mappings that an early greedy choice had excluded.

Relation to Prior Work

Two strands of prior work motivate the proposal by exposing the gap. BERTMap and its toolkit DeepOnto predict mappings with a fine-tuned Bio-Clinical BERT and then repair them post hoc, with no feedback to the matcher. BERTMap also needs an extreme 0.999 threshold to reach its best F1. The optimality study shows that optimal repair, which minimizes the total confidence of deleted mappings, beats greedy repair in F-measure. The gap is largest at low thresholds, which is exactly where a matcher would recover recall. Optimal repair, however, does not scale to SNOMED.

Other work supplies the method. ADMM gives us the split. The shadow-weight analysis predicts why a deterministic accumulator should beat random exploration. Greedy Hash and HashNet train continuous scores against discrete targets, using a penalty and a gradually sharpened approximation respectively. Our quadratic coupling is a penalty of the Greedy Hash kind. The difference is that the discrete target is a coherent set defined by axioms, not an independent sign of each output.

Two results constrain the design. BOLD-SAM shows that continuous proxies can fail to move discrete states once their magnitudes grow, which it calls the vanishing perturbation effect. We therefore monitor whether $u$ grows so large that decisions freeze, and we bound $u$ by clipping as a hedge. The interaction-based repair work shows that some errors are "shadowed": they stay hidden behind more specific correct mappings and surface only under probabilistic relaxation. A deterministic dual may never expose them, and we test that directly rather than assume it away.

Research Program

The system is built on DeepOnto's BERTMap implementation, using the KRR-Oxford/BERTMap code. Candidates come from BERTMap's subword index, and we keep them down to a much lower confidence floor than 0.999 so that recall is available. LogMap-Repair, called over the JPype bridge, serves as the z-step oracle. A full reasoner run with ELK, and HermiT where it is tractable, certifies coherence of the final output.

The analysis has three parts. First, we measure how much of the greedy-to-optimal gap the iterated z-step closes when the matcher is frozen. This isolates the repair-side effect. Second, we measure what training the matcher adds on top. Third, we measure the dynamics: how many decisions flip per round, whether $u$ saturates, and whether flips concentrate on mappings involved in many conflicts. The convergence of ADMM on a nonconvex set is not guaranteed, so we treat it as an empirical question. We report the number of rounds and the oscillation between rounds rather than claiming a theorem.

Baselines

The naive reference point is BERTMap with no repair at its best threshold. It shows how much incoherence a strong neural matcher produces and what F1 costs repair incurs.

The strongest existing pipeline is BERTMap with its built-in logic-based repair, as distributed in DeepOnto. We run it on identical candidate sets with the same Bio-Clinical BERT checkpoint, and we tune its threshold on the same validation split, so threshold search is not an advantage for either side. On FMA-SNOMED, BERTMap is reported to exceed AML by 1.4% and LogMap by 4.2% in F1. On the Disease Ontology task, DeepOnto reports an F-score of 0.874. No absolute FMA-SNOMED F1 is reported in the file, so we will reproduce BERTMap ourselves and treat our reproduced number as the bar. We also run LogMap and AML, using OAEI configurations, as non-neural references. On Anatomy the comparison point is AML's 0.941 F-measure from OAEI 2020.

For repair quality we run Alcomo Optimal and ELog wherever they finish. That covers Conference and the smaller LargeBio fragments. Following the optimality study, they are not expected to finish on SNOMED within five hours.

Three ablations isolate the key ingredient, which is the accumulated dual. The first is single-round self-training: repair once, fine-tune the matcher on the repaired output as pseudo-labels, and set $u=0$. The second runs the frozen-matcher iterated repair described above. The third replaces deterministic dual accumulation with random perturbation of the repair weights. This is the analogue of stochastic rounding and of relaxation-style exploration.

Every variant gets the same number of matcher gradient steps and the same total budget of wall-clock repair calls. The success criteria are as follows. Beating the BERTMap-plus-repair pipeline means higher F1 at zero unsatisfiable classes on FMA-NCI, FMA-SNOMED and FMA-SNOMED+, with the gain coming mostly from recall. Beating greedy repair means closing at least half of the greedy-to-optimal F-measure gap where the optimum is computable, at less than five times the greedy runtime.

Why not just…?

Why not just raise the threshold, as BERTMap already does at 0.999? A high threshold avoids conflicts by discarding most low-confidence candidates, including correct ones. The optimality study finds that repair matters most precisely at low thresholds. We plot F1 and unsatisfiable-class counts across thresholds for both systems. If the threshold-tuned baseline matches us at its best threshold, the idea has failed on that dataset, and we will report it.

Why not just run greedy Horn repair once, as LogMap does? It is fast and dominant on LargeBio according to OAEI 2020, but it scores below optimal repair in F-measure. Our frozen-matcher ablation is exactly "greedy repair, iterated with memory." It shows directly whether iteration alone closes the gap, independent of any neural training.

Why not just fine-tune the matcher on the repaired alignment? This is the real threat. It is cheap, it already provides feedback from logic to the matcher, and it may capture most of the benefit. The single-round self-training ablation is that method under the same budget. Our claim survives only if the accumulated dual adds a measurable gain beyond it. In particular, the dual should repair cases where the first greedy repair deleted a correct mapping, which self-training would instead reinforce. We count those cases explicitly against the reference alignment.

Why not just call an exact solver for the z-step? Where it finishes, we do, and it serves as the upper bound. But ELog and Alcomo Optimal did not finish on SNOMED within five hours. Rerunning such a solver every ADMM round is therefore infeasible at the scale where the problem matters.

Proposed Experiments

We use the OAEI Conference track, where optimal debuggers run and where the optimality study measured the effect of logical expressiveness. We use the Anatomy track and the Large BioMed tasks FMA-NCI, FMA-SNOMED and BERTMap's harder FMA-SNOMED+. We use the Disease Ontology task from DeepOnto as a further check against the reported 0.874 F-score. Metrics are precision, recall and F1 against the reference alignments. We also report the number of unsatisfiable classes under ELK, with HermiT where feasible, total runtime, and, on tractable tasks, the fraction of the greedy-to-optimal F-measure gap that is closed.

There are three decisive checks. First, on Conference and the FMA-NCI fragments, the frozen-matcher iterated repair should land strictly between greedy and optimal F-measure. If it equals greedy, the dual is not doing the work we claim. Second, on FMA-SNOMED and FMA-SNOMED+, full ADMM should beat both reproduced BERTMap-plus-repair and single-round self-training in F1 at zero unsatisfiable classes, and its best operating threshold should fall well below 0.999. Third, deterministic accumulation should beat the randomized-weight variant at equal budget, mirroring the shadow-weight result. If the randomized variant wins, the shadowing argument from the interaction-based repair work is the better description of this problem, and we would redirect toward a hybrid that adds bounded exploration to the dual.

A known risk is that repair deletes correct mappings and training then entrenches the error. We therefore report how many correct mappings are removed in each round, alongside the headline F1.