Matching two databases or ontologies means deciding which column or concept in one corresponds to which in the other. The hard cases are rarely about spelling. Two columns can share almost every surface feature and still differ in one decisive respect. visit_start_datetime and visit_end_datetime differ in temporal role. A salary in dollars and a salary in thousands of dollars differ in unit. "Lung cancer" and "cancer" differ in hierarchy level. Current matchers find candidate matches with a general-purpose text embedding and then ask a large language model (LLM) to choose among them. Because that embedding was never trained to separate such near-misses, the right answer ends up buried in a large pool, and the LLM's choices degrade as the pool grows.
We propose to train the candidate retriever on synthetic near-misses that are typed. Each negative is a counterfactual copy of a real column or concept in which an LLM, or a programmatic transform on sampled data values, changes exactly one semantic axis: temporal or causal role, unit or scale, aggregation, granularity, or hierarchy direction. Every other feature stays the same. The idea is borrowed from speech keyword spotting. There, systematic edits at a controlled distance from the target word produced confusable training examples that generalized to real confusions. We replace letter edits with edits along named axes of semantic heterogeneity, and we test whether this typing, rather than synthesis alone, is what tightens the candidate pool on real benchmarks.
Core Claim
Semantic heterogeneity is not unstructured noise. Most damaging confusions fall along a small number of recurring axes. A retriever trained with negatives that each isolate one axis should learn a boundary along each of them. Two outcomes follow. Recall at small pool sizes should rise, so a downstream LLM selector sees about 5 to 10 candidates instead of 50. And errors become diagnosable by axis.
The claim is falsifiable against a strong, cheap competitor: hard negatives mined from the target schema itself. Target schemas such as OMOP already contain many natural siblings. If mined siblings close the gap equally well, synthesis is unnecessary, and we will report that. Our specific bet is that mining cannot produce the unit, scale and granularity counterfactuals that a given schema happens not to contain, and cannot label which axis a negative tests. Typed synthesis can do both.
Where This Sits in Prior Work
Work that supplies the method. Three keyword-spotting papers show the recipe of controlled confusable generation.
- GraphemeAug recursively adds, removes and substitutes letters at a set edit distance and synthesizes the results as speech. It reports a 54% AUC gain on real confusables. It also warns that very close negatives raise false rejections of true keywords.
- LLM-Synth4KWS prompts an LLM for words that share a target sound. It finds that synthetic data introduces distribution bias at very low false-accept rates.
- SyNeg carries LLM-written hard negatives into passage retrieval. It reports a gain of about 3 NDCG@10 points on BEIR, but only when synthetic negatives are mixed with mined ones at roughly 0.7 per positive. Training on synthetic negatives alone was unstable.
We reuse three things from this work: the controlled-distance edits, the attention to synthetic-to-real transfer, and the mixing ratio. What changes is the target. Our edits act on structured column and concept descriptions and on data values, and each edit is labeled by axis. None of these three papers labels its negatives by the kind of difference they encode.
Work that motivates the gap.
- Enhancing Biomedical Schema Matching with LLM-based Training Data Generation trains a contrastive column encoder, meaning one that pulls matching columns together and pushes others apart. It uses LLM-generated positive pairs only, and it observes that its LLM selector loses precision on 50-candidate pools. This is the closest prior system. It generates positives; we generate typed negatives.
- Matchmaker treats matching as retrieval plus LLM ranking with an abstain option. It reaches 62.2% accuracy@1 on MIMIC-OMOP, but it uses frozen ColBERT-v2 embeddings and reads metadata only, never values.
- GenOM retrieves with text-embedding-3-small over LLM-written concept definitions.
- LogMapLLM uses an LLM only as a post-hoc oracle on low-confidence pairs.
Both ontology systems handle equivalence only and leave subsumption ("is-a") open. That opening is exactly what our hierarchy-direction axis targets.
Work that constrains the design. - Towards a Benchmark for Instance Matching documents failures when the same entity is recorded at different granularities. This motivates a granularity axis. - Dimensional enrichment of statistical linked open data documents failures when the same metric uses different units or currency scales. This motivates a unit axis that must reach data values, not just names.
Research Program
The generator takes an unlabeled source or target element: its name, description, parent table or ancestor chain, and, where available, a sample of values. It produces a typed counterfactual. Metadata edits come from an LLM prompted with one axis and an instruction to keep everything else fixed. In the spirit of SyNeg, the LLM must also state why the counterfactual looks equivalent but is not.
Value edits are programmatic and need no LLM:
- rescaling a numeric sample by a factor of 1000 or by a unit conversion;
- replacing timestamps with a shifted companion event;
- collapsing a structured address into a single string, or splitting it apart;
- turning per-row values into a group count or sum.
Hierarchy negatives come directly from the ontology. A concept's parent and children serve as negatives for equivalence. A separate head labels them as broader or narrower.
The number of axes edited plays the role of GraphemeAug's edit distance. One-axis negatives are hardest. Two- and three-axis negatives provide a curriculum. Positives come from SSM-style LLM paraphrases. No gold mapping is used anywhere, so the method stays zero-shot in the same sense as Matchmaker.
The retriever is a dual encoder initialized from the Matchmaker and GenOM retrieval backbones. It is trained with a contrastive loss over three negative sources: in-batch, mined, and typed-synthetic. We sweep the share of synthetic negatives around SyNeg's 0.7 and log the variance of gradients to check whether the instability SyNeg reported also appears here.
A filtering step guards against false negatives, meaning counterfactuals that are actually still equivalent. An independent LLM judge rejects any counterfactual it considers equivalent to its source. We audit a few hundred rejected and accepted examples by hand.
Baselines
Naive reference. The unmodified retriever each pipeline already uses: ColBERT-v2 for the Matchmaker pipeline and text-embedding-3-small for the GenOM pipeline. This is the starting point any fine-tuning must improve on.
Strongest existing schema system. Matchmaker as published. It reports accuracy@1 of 62.2% on MIMIC-OMOP and 70.2% on Synthea-OMOP. ReMatch reaches 42.5% on MIMIC-OMOP. Our comparison keeps Matchmaker's ranking stage, LLM and bootstrapping procedure fixed and swaps only the retriever. Beating it means accuracy@1 above 62.2% on MIMIC-OMOP with equal LLM calls, or equal accuracy with a pool at least half the size.
Strongest existing ontology system. LogMapLLM, which raised OAEI 2025 Bio-ML average F-score from 0.718 to 0.751. Alongside it we run GenOM, whose file-reported gain is relative: 35.3% mean F1 over LLM4OM. In the GenOM pipeline we replace its frozen retriever with ours and keep its 32B judge and the LogMapLt fusion step. Beating LogMapLLM means average F-score above 0.751 on the same five tasks.
Positive-only training. The SSM recipe, trained on identical positives and compute. On CPTAC-GDC the file reports only that SSM with LLM selection won top-1 precision on 7 of 8 tables. No reported numeric top-1 result was found on Lacuna for CPTAC-GDC, so we will rerun it ourselves.
Mined hard negatives. Same encoder, same budget, with negatives drawn from target-schema siblings, BM25, Levenshtein neighbors and the embedding's own nearest neighbors. This is the main threat.
Untyped synthetic negatives. SyNeg-style prompting, "write a column that looks matching but is not," with the same LLM, count and mixing ratio. This ablation isolates typing.
We also run leave-one-axis-out ablations and a variant without value-level transforms. All variants receive the same number of negatives, training steps and hyperparameter search budget, with three seeds each.
Why not just…?
Why not just rerank a larger pool with a stronger LLM, as Matchmaker or LogMapLLM do? SSM reports that LLM selection precision drops as pools reach 50 candidates. Pairwise LLM calls also scale with pool size times schema size. Still, a stronger reranker is a real possibility. We will plot accuracy@1 against pool size k for the frozen and trained retrievers under the same selector. If the curves converge at k=50 with no cost difference, our contribution reduces to efficiency, and we will say so.
Why not just mine hard negatives from the target schema? This is the most serious alternative. OMOP and SNOMED are dense with natural siblings, and mining is free. The evidence that synthesis beats mining comes from speech (GraphemeAug) and from passages, where BM25 mining introduces false negatives (SyNeg). Neither setting is schemas. Our error analysis is stratified by axis, so it can separate the cases. If mining matches us on temporal-role and sibling errors but loses on unit, scale and granularity errors, typed synthesis is justified only for those axes. If mining matches everywhere, the hypothesis fails.
Why not just let an LLM write generic hard negatives, as SyNeg does? That is the untyped ablation above. If it matches typed negatives, the axis structure adds nothing beyond prompt variety.
Why not just use lexical matchers like LogMap or COMA? The file shows they already fall short. LogMap's own confidence heuristic reaches a Youden's index of only about 0.094 on ambiguous pairs (LogMapLLM). Coma, Cupid and Similarity Flooding lose to learned embeddings on CPTAC-GDC (SSM). We keep LogMapLt only as a fusion component, matching GenOM.
Proposed Experiments
We evaluate schema matching on MIMIC-OMOP, Synthea-OMOP, CPTAC-GDC, Magellan and WikiData, and ontology matching on OAEI 2024 and 2025 Bio-ML. For Magellan and WikiData, no reported numeric result was found on Lacuna, so we rerun Matchmaker ourselves. Value-level transforms apply only where row values are available: Synthea and CPTAC directly, and MIMIC under its access terms. Metadata-only datasets test the metadata axes alone.
Metrics are recall@k for k in {1, 5, 10, 50}, end-to-end accuracy@1 or F-score, and LLM calls per matched element.
First decisive check: real confusions, not synthetic ones. Following GraphemeAug's use of eval-real-conf, we label each baseline's highest-ranked wrong candidate on MIMIC-OMOP and Synthea-OMOP by axis, for example visit_start against visit_end. We report the error rate per axis. Gains measured only on our own counterfactuals would be dismissed as circular.
Second decisive check: stability and the cost of close negatives. We sweep the synthetic share from 0 to 1 and track false rejections, meaning true matches pushed out of the top-k. This tests whether SyNeg's instability and GraphemeAug's false-rejection effect carry over to schemas.
Third decisive check: subsumption. We test whether the hierarchy head separates parent, child and sibling concepts. Bio-ML subsumption references are used where the track supplies them. Otherwise we build references from held-out ontology edges. This is the direction where GenOM and LogMapLLM explicitly stop.
The proposal succeeds if the trained retriever reaches the same end-to-end accuracy with a much smaller pool, beats mined negatives on at least the unit, scale and granularity axes, and beats untyped synthesis. Failing either of the last two comparisons would show that typing or synthesis is not doing the work.
References
- GraphemeAug: A Systematic Approach to Synthesized Hard Negative Keyword Spotting Examples
- LLM-Synth4KWS: Scalable Automatic Generation and Synthesis of Confusable Data for Custom Keyword Spotting
- SyNeg: LLM-Driven Synthetic Hard-Negatives for Dense Retrieval
- Enhancing Biomedical Schema Matching with LLM-based Training Data Generation
- Bootstrapping Self-Improvement of Language Model Programs for Zero-Shot Schema Matching
- GenOM: Ontology Matching with Description Generation and Large Language Model
- Large Language Models as Oracles for Ontology Alignment
- Towards a Benchmark for Instance Matching
- Dimensional enrichment of statistical linked open data